Database indexing in performance measurement systems

ABSTRACT

A performance measurement indexing system indexes a data store containing data entries indicative of message processing by an application. The application includes a plurality of checkpoints, and the data store contains data logged upon each message traversing the checkpoints in the application. The performance measurement indexing system determines which data entries relate to messages that satisfy a delay condition, and limits queries run on the data store to those data entries, thereby increasing the speed and efficiency with which queries can be serviced.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation under 37 C.F.R. § 1.53(b) of U.S.patent application Ser. No. 17/015,184 filed Sep. 9, 2020, now U.S. Pat.No. ______, which is a continuation under 37 C.F.R. § 1.53(b) of U.S.patent application Ser. No. 15/726,922 filed Oct. 6, 2017, now U.S. Pat.No. 10,803,042, the entirety of all of which are incorporated byreference herein and relied upon.

BACKGROUND

Performance measurement systems monitor and record information about theoperation of a system, such as . . . by logging the progress of messagesthrough an application or code. Example information that is oftenlogged/recorded by such systems includes data indicative of theoccurrence of an event, such as when a processing of a particularmessage began or was completed by a particular processing stage, andtime information about when that event occurred. The logged data may bestored in a data store as the data is collected. The data store may bequeried, e.g., during a post-processing phase, by a performance analysistool to derive performance metrics about the application, e.g., thespeed, efficiency, etc. with which the application processed themessages. A high volume data transaction processing system may process,and accordingly record the details for, many messages/transactions. Theapplication performance data becomes more useful as more data iscollected/analyzed. However, too many data entries in the data store candrastically increase query response time.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts a computer network system, according to some embodiments.

FIG. 2 depicts a general computer system, according to some embodiments.

FIG. 3A depicts a storage data structure, according to some embodiments.

FIG. 3B depicts an alternative storage data structure, according to someembodiments.

FIG. 3C depicts an order book data structure, according to someembodiments.

FIG. 4 depicts a match engine module, according to some embodiments.

FIG. 5 depicts an indexing system implemented in a data transactionprocessing system, according to some embodiments.

FIG. 6 depicts an example application including a plurality ofcheckpoints, according to some embodiments.

FIG. 7A depicts an example data store including data entries indicativeof message processing in a data transaction processing system, accordingto some embodiments.

FIG. 7B depicts example message processing latencies for messagesprocessed in a data transaction processing system, according to someembodiments.

FIG. 7C depicts an example indexed data store including data entriesindicative of message processing in a data transaction processingsystem, according to some embodiments.

FIG. 8 depicts a high-level flowchart illustrating a method for indexinga data store, according to some embodiments.

DETAILED DESCRIPTION

The disclosed embodiments generally relate to methods and systems forindexing a data store so that the time required to query the data storeis reduced. In particular, the disclosed embodiments may be implementedas a performance measurement indexing system that parses a data storestoring data entries recorded or logged by a monitoring system, andreduces the dataset retrieved/searched in response to queries on thedata store.

A data transaction processing system may include an application thatprocesses messages or transactions. A monitoring system records monitorsprocessing of messages in a data transaction processing system bylogging information, i.e., tracer entries, about those messages as theyare processed through the application/data transaction processingsystem, i.e., as they are processed at/through various “checkpoints” inthe system.

An application may include a plurality of checkpoints. Checkpoints maybe inserted at specified locations or lines in the application code by adeveloper, e.g., at points of interest in the code. A checkpoint may beassociated with a line in the code. A checkpoint's location in the codemay be identifiable, e.g., via an identifier for the associated line, ora line number. At each checkpoint, the software code may include amethod call to log information in the data store. When a messagetraverses a checkpoint, the monitoring system logs information about theoccurrence of the traversal event by recording data indicative of themessage, the checkpoint name, and a timestamp associated with theoccurrence of the event.

During a post-processing phase, the performance measurement indexingsystem determines the latencies associated with various checkpointpairs, and marks or indicates which areas of the application areassociated with the highest latencies, or is congested. For example, theperformance measurement indexing system may include a latency detectionsystem that determines whether a portion of a code experienced a higherthan expected, or lower than expected, latency. When the data store isqueried, e.g., by a performance analysis tool, the performancemeasurement indexing system enables the performance analysis tool toonly retrieve/search the portions of the application that werecongested.

The disclosed embodiments improve the speed with which performanceanalysis tool queries can be performed on the data store, which containsmessage processing data. The performance measurement indexing systemfocuses the field of the search to just the portions of the applicationthat are associated with unexpected/undesirable delays.

The disclosed embodiments also improve the efficiency and convenience ofthe user operation. A user can perform a query and obtain query resultsquickly because the performance measurement indexing system has alreadylimited the field of the search to the areas that should be of mostinterest to the performance analysis tool, the areas of the applicationsuffering from unexpected performance delays.

The disclosed embodiments also improve upon the technical field ofperformance measurement database search and retrieval by formatting thedata store for the specific type of tool performing a query on the datastore. At least some of the problems solved by the disclosed embodimentsare specifically rooted in technology, where a data store collectshigh-fidelity, granular data for millions of transactions causingexceedingly large data store sizes, which result in time consumingqueries, and are solved by the disclosed performance measurementindexing system that determines latencies between combinations ofcheckpoints during a post-processing phase, before a performanceanalysis tools runs queries on the data store, and makes the results ofthe indexing available to the performance analysis tool queries.

Some database indexing methods, which are configured to work withqueries to search for all instances of specified keys/values within thedatabase, enable fast responses by generating, ahead of time (e.g.,before a query is received), a separate data structure that stores eachunique key with pointers to all of the instances of the key, so thatwhen such queries are received, the queries can be answered quicklyusing the data structure. However, the disclosed embodiments are alsoapplicable to queries that do not request a specific key or value to besearched in the data store. For example, the disclosed embodiments canbe implemented to enable faster searching of performance measurementdata, where the indexing is not specific to the searched keys/values.

Exchange Computing System

The disclosed embodiments may be implemented in a data transactionprocessing system that processes data items or objects. Customer or userdevices (e.g., client computers) may submit electronic data transactionrequest messages, e.g., inbound messages, to the data transactionprocessing system over a data communication network. The electronic datatransaction request messages may include, for example, transactionmatching parameters, such as instructions and/or values, for processingthe data transaction request messages within the data transactionprocessing system. The instructions may be to perform transactions,e.g., buy or sell a quantity of a product at a range of values definedequations. Products, e.g., financial instruments, or order booksrepresenting the state of an electronic marketplace for a product, maybe represented as data objects within the exchange computing system. Theinstructions may also be conditional, e.g., buy or sell a quantity of aproduct at a given value if a trade for the product is executed at someother reference value. The data transaction processing system mayinclude various specifically configured matching processors that match,e.g., automatically, electronic data transaction request messages forthe same one of the data items or objects. The specifically configuredmatching processors may match, or attempt to match, electronic datatransaction request messages based on multiple transaction matchingparameters from the different client computers. The specificallyconfigured matching processors may additionally generate informationindicative of a state of an environment (e.g., the state of the orderbook) based on the processing, and report this information to datarecipient computing systems via outbound messages published via one ormore data feeds.

For example, one exemplary environment where the disclosed embodimentsmay be desirable is in financial markets, and in particular, electronicfinancial exchanges, such as a futures exchange, such as the ChicagoMercantile Exchange Inc. (CME).

A financial instrument trading system, such as a futures exchange, suchas the Chicago Mercantile Exchange Inc. (CME), provides a contractmarket where financial instruments, e.g., futures and options onfutures, are traded using electronic systems. “Futures” is a term usedto designate all contracts for the purchase or sale of financialinstruments or physical commodities for future delivery or cashsettlement on a commodity futures exchange. A futures contract is alegally binding agreement to buy or sell a commodity at a specifiedprice at a predetermined future time. An option contract is the right,but not the obligation, to sell or buy the underlying instrument (inthis case, a futures contract) at a specified price on or before acertain expiration date. An option contract offers an opportunity totake advantage of futures price moves without actually having a futuresposition. The commodity to be delivered in fulfillment of the contract,or alternatively the commodity for which the cash market price shalldetermine the final settlement price of the futures contract, is knownas the contract's underlying reference or “underlier.” The underlying orunderlier for an options contract is the corresponding futures contractthat is purchased or sold upon the exercise of the option.

The terms and conditions of each futures contract are standardized as tothe specification of the contract's underlying reference commodity, thequality of such commodity, quantity, delivery date, and means ofcontract settlement. Cash settlement is a method of settling a futurescontract whereby the parties effect final settlement when the contractexpires by paying/receiving the loss/gain related to the contract incash, rather than by effecting physical sale and purchase of theunderlying reference commodity at a price determined by the futurescontract, price. Options and futures may be based on more generalizedmarket indicators, such as stock indices, interest rates, futurescontracts and other derivatives.

An exchange may provide for a centralized “clearing house” through whichtrades made must be confirmed, matched, and settled each day untiloffset or delivered. The clearing house may be an adjunct to anexchange, and may be an operating division of an exchange, which isresponsible for settling trading accounts, clearing trades, collectingand maintaining performance bond funds, regulating delivery, andreporting trading data. One of the roles of the clearing house is tomitigate credit risk. Clearing is the procedure through which theclearing house becomes buyer to each seller of a futures contract, andseller to each buyer, also referred to as a novation, and assumesresponsibility for protecting buyers and sellers from financial loss dueto breach of contract, by assuring performance on each contract. Aclearing member is a firm qualified to clear trades through the clearinghouse.

An exchange computing system may operate under a central counterpartymodel, where the exchange acts as an intermediary between marketparticipants for the transaction of financial instruments. Inparticular, the exchange computing system novates itself into thetransactions between the market participants, i.e., splits a giventransaction between the parties into two separate transactions where theexchange computing system substitutes itself as the counterparty to eachof the parties for that part of the transaction, sometimes referred toas a novation. In this way, the exchange computing system acts as aguarantor and central counterparty and there is no need for the marketparticipants to disclose their identities to each other, or subjectthemselves to credit or other investigations by a potentialcounterparty. For example, the exchange computing system insulates onemarket participant from the default by another market participant.Market participants need only meet the requirements of the exchangecomputing system. Anonymity among the market participants encourages amore liquid market environment as there are lower barriers toparticipation. The exchange computing system can accordingly offerbenefits such as centralized and anonymous matching and clearing.

Electronic Data Transaction Request Messages

As used herein, a financial message, or an electronic message, refersboth to messages communicated by market participants to an electronictrading or market system and vice versa. The messages may becommunicated using packeting or other techniques operable to communicateinformation between systems and system components. Some messages may beassociated with actions to be taken in the electronic trading or marketsystem. In particular, in one embodiment, upon receipt of a request, atoken is allocated and included in a TCP shallow acknowledgmenttransmission sent back to the participant acknowledging receipt of therequest. It should be appreciated that while this shallow acknowledgmentis, in some sense, a response to the request, it does not confirm theprocessing of an order included in the request. The participant, i.e.,their device, then sends back a TCP acknowledgment which acknowledgesreceipt of the shallow acknowledgment and token.

Financial messages communicated to the electronic trading system, alsoreferred to as “inbound” messages, may include associated actions thatcharacterize the messages, such as trader orders, order modifications,order cancellations and the like, as well as other message types.Inbound messages may be sent from client devices associated with marketparticipants, or their representatives, e.g., trade order messages,etc., to an electronic trading or market system. For example, a marketparticipant may submit an electronic message to the electronic tradingsystem that includes an associated specific action to be undertaken bythe electronic trading system, such as entering a new trade order intothe market or modifying an existing order in the market. In oneembodiment, if a participant wishes to modify a previously sent request,e.g., a prior order which has not yet been processed or traded, they maysend a request message comprising a request to modify the prior request.In one exemplary embodiment, the incoming request itself, e.g., theinbound order entry, may be referred to as an iLink message. iLink is abidirectional communications/message protocol/message format implementedby the Chicago Mercantile Exchange Inc.

Financial messages communicated from the electronic trading system,referred to as “outbound” messages, may include messages responsive toinbound messages, such as confirmation messages, or other messages suchas market update messages, quote messages, and the like. Outboundmessages may be disseminated via data feeds.

Financial messages may further be categorized as having or reflecting animpact on a market or electronic marketplace, also referred to as an“order book” or “book,” for a traded product, such as a prevailing pricetherefore, number of resting orders at various price levels andquantities thereof, etc., or not having or reflecting an impact on amarket or a subset or portion thereof. In one embodiment, an electronicorder book may be understood to be an electronic collection of theoutstanding or resting orders for a financial instrument.

For example, a request to place a trade may result in a responseindicative of the trade either being matched with, or being rested on anorder book to await, a suitable counter-order. This response may includea message directed solely to the trader who submitted the order toacknowledge receipt of the order and report whether it was matched, andthe extent thereto, or rested. The response may further include amessage to all market participants reporting a change in the order bookdue to the order. This response may take the form of a report of thespecific change to the order book, e.g., an order for quantity X atprice Y was added to the book (referred to, in one embodiment, as aMarket By Order message), or may simply report the result, e.g., pricelevel Y now has orders for a total quantity of Z (where Z is the sum ofthe previous resting quantity plus quantity X of the new order). In somecases, requests may elicit a non-impacting response, such as temporallyproximate to the receipt of the request, and then cause a separatemarket-impact reflecting response at a later time. For example, a stoporder, fill or kill order (FOK), also known as an immediate or cancelorder, or other conditional request may not have an immediate marketimpacting effect, if at all, until the requisite conditions are met.

An acknowledgement or confirmation of receipt, e.g., a non-marketimpacting communication, may be sent to the trader simply confirmingthat the order was received. Upon the conditions being met and a marketimpacting result thereof occurring, a market-impacting message may betransmitted as described herein both directly back to the submittingmarket participant and to all market participants (in a Market By Price“MBP” e.g., Aggregated By Value (“ABV”) book, or Market By Order “MBO”,e.g., Per Order (“PO”) book format). It should be appreciated thatadditional conditions may be specified, such as a time or price limit,which may cause the order to be dropped or otherwise canceled and thatsuch an event may result in another non-market-impacting communicationinstead. In some implementations, market impacting communications may becommunicated separately from non-market impacting communications, suchas via a separate communications channel or feed.

It should be further appreciated that various types of market data feedsmay be provided which reflect different markets or aspects thereof.Market participants may then, for example, subscribe to receive thosefeeds of interest to them. For example, data recipient computing systemsmay choose to receive one or more different feeds. As market impactingcommunications usually tend to be more important to market participantsthan non-impacting communications, this separation may reduce congestionand/or noise among those communications having or reflecting an impacton a market or portion thereof. Furthermore, a particular market datafeed may only communicate information related to the top buy/sell pricesfor a particular product, referred to as “top of book” feed, e.g., onlychanges to the top 10 price levels are communicated. Such limitationsmay be implemented to reduce consumption of bandwidth and messagegeneration resources. In this case, while a request message may beconsidered market-impacting if it affects a price level other than thetop buy/sell prices, it will not result in a message being sent to themarket participants.

Examples of the various types of market data feeds which may be providedby electronic trading systems, such as the CME, in order to providedifferent types or subsets of market information or to provide suchinformation in different formats include Market By Order or Per Order,Market Depth (also known as Market by Price or Aggregated By Value to adesignated depth of the book), e.g., CME offers a 10-deep market byprice feed, Top of Book (a single depth Market by Price feed), andcombinations thereof. There may also be all manner of specialized feedsin terms of the content, i.e., providing, for example, derived data,such as a calculated index.

Market data feeds may be characterized as providing a “view” or“overview” of a given market, an aggregation or a portion thereof orchanges thereto. For example, a market data feed, such as a Market ByPrice (“MBP”) feed, also known as an Aggregated By Value (“ABV”) feed,may convey, with each message, the entire/current state of a market, orportion thereof, for a particular product as a result of one or moremarket impacting events. For example, an MBP message may convey a totalquantity of resting buy/sell orders at a particular price level inresponse to a new order being placed at that price. An MBP message mayconvey a quantity of an instrument which was traded in response to anincoming order being matched with one or more resting orders. MBPmessages may only be generated for events affecting a portion of amarket, e.g., only the top 10 resting buy/sell orders and, thereby, onlyprovide a view of that portion. As used herein, a market impactingrequest may be said to impact the “view” of the market as presented viathe market data feed.

An MBP feed may utilize different message formats for conveyingdifferent types of market impacting events. For example, when a neworder is rested on the order book, an MBP message may reflect thecurrent state of the price level to which the order was added, e.g., thenew aggregate quantity and the new aggregate number of resting orders.As can be seen, such a message conveys no information about theindividual resting orders, including the newly rested order, themselvesto the market participants. Only the submitting market participant, whoreceives a separate private message acknowledging the event, knows thatit was their order that was added to the book. Similarly, when a tradeoccurs, an MBP message may be sent which conveys the price at which theinstrument was traded, the quantity traded and the number ofparticipating orders, but may convey no information as to whoseparticular orders contributed to the trade. MBP feeds may further batchreporting of multiple events, i.e., report the result of multiple marketimpacting events in a single message.

Alternatively, a market data feed, referred to as a Market By Order(“MBO”) feed also known as a Per Order (“PO”) feed, may convey datareflecting a change that occurred to the order book rather than theresult of that change, e.g., that order ABC for quantity X was added toprice level Y or that order ABC and order XYZ traded a quantity X at aprice Y. In this case, the MBO message identifies only the change thatoccurred so a market participant wishing to know the current state ofthe order book must maintain their own copy and apply the changereflected in the message to know the current state. As can be seen,MBO/PO messages may carry much more data than MBP/ABV messages becauseMBO/PO messages reflect information about each order, whereas MBP/ABVmessages contain information about orders affecting some predeterminedvalue levels. Furthermore, because specific orders, but not thesubmitting traders thereof, are identified, other market participantsmay be able to follow that order as it progresses through the market,e.g., as it is modified, canceled, traded, etc.

An ABV book data object may include information about multiple values.The ABV book data object may be arranged and structured so thatinformation about each value is aggregated together. Thus, for a givenvalue V, the ABV book data object may aggregate all the information byvalue, such as for example, the number of orders having a certainposition at value V, the quantity of total orders resting at value V,etc. Thus, the value field may be the key, or may be a unique field,within an ABV book data object. In one embodiment, the value for eachentry within the ABV book data object is different. In one embodiment,information in an ABV book data object is presented in a manner suchthat the value field is the most granular field of information.

A PO book data object may include information about multiple orders. ThePO book data object may be arranged and structured so that informationabout each order is represented. Thus, for a given order O, the PO bookdata object may provide all of the information for order O. Thus, theorder field may be the key, or may be a unique field, within a PO bookdata object. In one embodiment, the order ID for each entry within thePO book data object is different. In one embodiment, information in a PObook data object is presented in a manner such that the order field isthe most granular field of information.

Thus, the PO book data object may include data about unique orders,e.g., all unique resting orders for a product, and the ABV book dataobject may include data about unique values, e.g., up to a predeterminedlevel, e.g., top ten price or value levels, for a product.

It should be appreciated that the number, type and manner of market datafeeds provided by an electronic trading system are implementationdependent and may vary depending upon the types of products traded bythe electronic trading system, customer/trader preferences, bandwidthand data processing limitations, etc. and that all such feeds, nowavailable or later developed, are contemplated herein. MBP/ABV andMBO/PO feeds may refer to categories/variations of market data feeds,distinguished by whether they provide an indication of the current stateof a market resulting from a market impacting event (MBP) or anindication of the change in the current state of a market due to amarket impacting event (MBO).

Messages, whether MBO or MBP, generated responsive to market impactingevents which are caused by a single order, such as a new order, an ordercancellation, an order modification, etc., are fairly simple and compactand easily created and transmitted. However, messages, whether MBO orMBP, generated responsive to market impacting events which are caused bymore than one order, such as a trade, may require the transmission of asignificant amount of data to convey the requisite information to themarket participants. For trades involving a large number of orders,e.g., a buy order for a quantity of 5000 which matches 5000 sell orderseach for a quantity of 1, a significant amount of information may needto be sent, e.g., data indicative of each of the 5000 trades that haveparticipated in the market impacting event.

In one embodiment, an exchange computing system may generate multipleorder book objects, one for each type of view that is published orprovided. For example, the system may generate a PO book object and anABV book object. It should be appreciated that each book object, or viewfor a product or market, may be derived from the Per Order book object,which includes all the orders for a given financial product or market.

An inbound message may include an order that affects the PO book object,the ABV book object, or both. An outbound message may include data fromone or more of the structures within the exchange computing system,e.g., the PO book object queues or the ABV book object queues.

Furthermore, each participating trader needs to receive a notificationthat their particular order has traded. Continuing with the example,this may require sending 5001 individual trade notification messages, oreven 10,000+ messages where each contributing side (buy vs. sell) isseparately reported, in addition to the notification sent to all of themarket participants.

As detailed in U.S. Patent Publication No. 2015/0161727, the entirety ofwhich is incorporated by reference herein and relied upon, it may berecognized that trade notifications sent to all market participants mayinclude redundant information repeated for each participating trade anda structure of an MBP trade notification message may be provided whichresults in a more efficient communication of the occurrence of a trade.The message structure may include a header portion which indicates thetype of transaction which occurred, i.e., a trade, as well as othergeneral information about the event, an instrument portion whichcomprises data about each instrument which was traded as part of thetransaction, and an order portion which comprises data about eachparticipating order. In one embodiment, the header portion may include amessage type, Transaction Time, Match Event Indicator, and Number ofMarket Data Entries (“No. MD Entries”) fields. The instrument portionmay include a market data update action indicator (“MD Update Action”),an indication of the Market Data Entry Type (“MD Entry Type”), anidentifier of the instrument/security involved in the transaction(“Security ID”), a report sequence indicator (“Rpt Seq”), the price atwhich the instrument was traded (“MD Entry PX”), the aggregate quantitytraded at the indicated price (“ConsTradeQty”), the number ofparticipating orders (“NumberOfOrders”), and an identifier of theaggressor side (“Aggressor Side”) fields. The order portion may furtherinclude an identifier of the participating order (“Order ID”), describedin more detail below, and the quantity of the order traded (“MD EntrySize”) fields. It should be appreciated that the particular fieldsincluded in each portion are implementation dependent and that differentfields in addition to, or in lieu of, those listed may be includeddepending upon the implementation. It should be appreciated that theexemplary fields can be compliant with the FIX binary and/or FIX/FASTprotocol for the communication of the financial information.

The instrument portion contains a set of fields, e.g., seven fieldsaccounting for 23 bytes, which are repeated for each participatinginstrument. In complex trades, such as trades involving combinationorders or strategies, e.g., spreads, or implied trades, there may bemultiple instruments being exchanged among the parties. In oneembodiment, the order portion includes only one field, accounting for 4bytes, for each participating order which indicates the quantity of thatorder which was traded. As will be discussed below, the order portionmay further include an identifier of each order, accounting for anadditional 8 bytes, in addition to the quantity thereof traded. Asshould be appreciated, data which would have been repeated for eachparticipating order, is consolidated or otherwise summarized in theheader and instrument portions of the message thereby eliminatingredundant information and, overall, significantly reducing the size ofthe message.

While the disclosed embodiments may be discussed with respect to an MBPmarket data feed, it should be appreciated that the disclosedembodiments may also be applicable to an MBO market data feed.

Market Segment Gateway

In one embodiment, the disclosed system may include a Market SegmentGateway (“MSG”) that is the point of ingress/entry and/oregress/departure for all transactions, i.e., the network traffic/packetscontaining the data therefore, specific to a single market at which theorder of receipt of those transactions may be ascribed. An MSG or MarketSegment Gateway may be utilized for the purpose of deterministicoperation of the market. The electronic trading system may includemultiple markets, and because the electronic trading system includes oneMSG for each market/product implemented thereby, the electronic tradingsystem may include multiple MSGs. For more detail on deterministicoperation in a trading system, see U.S. Patent Publication No.2015/0127513 entitled “Transactionally Deterministic High SpeedFinancial Exchange Having Improved, Efficiency, Communication,Customization, Performance, Access, Trading Opportunities, CreditControls, And Fault Tolerance” and filed on Nov. 7, 2013 (“the '513Publication”), the entire disclosure of which is incorporated byreference herein and relied upon.

For example, a participant may send a request for a new transaction,e.g., a request for a new order, to the MSG. The MSG extracts or decodesthe request message and determines the characteristics of the requestmessage.

The MSG may include, or otherwise be coupled with, a buffer, cache,memory, database, content addressable memory, data store or other datastorage mechanism, or combinations thereof, which stores data indicativeof the characteristics of the request message. The request is passed tothe transaction processing system, e.g., the match engine.

An MSG or Market Segment Gateway may be utilized for the purpose ofdeterministic operation of the market. Transactions for a particularmarket may be ultimately received at the electronic trading system viaone or more points of entry, e.g., one or more communicationsinterfaces, at which the disclosed embodiments apply determinism, whichas described may be at the point where matching occurs, e.g., at eachmatch engine (where there may be multiple match engines, each for agiven product/market, or moved away from the point where matching occursand closer to the point where the electronic trading system firstbecomes “aware” of the incoming transaction, such as the point wheretransaction messages, e.g., orders, ingress the electronic tradingsystem. Generally, the terms “determinism” or “transactionaldeterminism” may refer to the processing, or the appearance thereof, oforders in accordance with defined business rules. Accordingly, as usedherein, the point of determinism may be the point at which theelectronic trading system ascribes an ordering to incomingtransactions/orders relative to other incoming transactions/orders suchthat the ordering may be factored into the subsequent processing, e.g.,matching, of those transactions/orders as will be described. For moredetail on deterministic operation in a trading system, see the '513Publication.

Electronic Trading

Electronic trading of financial instruments, such as futures contracts,is conducted by market participants sending orders, such as to buy orsell one or more futures contracts, in electronic form to the exchange.These electronically submitted orders to buy and sell are then matched,if possible, by the exchange, i.e., by the exchange's matching engine,to execute a trade. Outstanding (unmatched, wholly unsatisfied/unfilledor partially satisfied/filled) orders are maintained in one or more datastructures or databases referred to as “order books,” such orders beingreferred to as “resting,” and made visible, i.e., their availability fortrading is advertised, to the market participants through electronicnotifications/broadcasts, referred to as market data feeds. An orderbook is typically maintained for each product, e.g., instrument, tradedon the electronic trading system and generally defines or otherwiserepresents the state of the market for that product, i.e., the currentprices at which the market participants are willing buy or sell thatproduct. As such, as used herein, an order book for a product may alsobe referred to as a market for that product.

Upon receipt of an incoming order to trade in a particular financialinstrument, whether for a single-component financial instrument, e.g., asingle futures contract, or for a multiple-component financialinstrument, e.g., a combination contract such as a spread contract, amatch engine, as described herein, will attempt to identify a previouslyreceived but unsatisfied order counter thereto, i.e., for the oppositetransaction (buy or sell) in the same financial instrument at the sameor better price (but not necessarily for the same quantity unless, forexample, either order specifies a condition that it must be entirelyfilled or not at all).

Previously received but unsatisfied orders, i.e., orders which eitherdid not match with a counter order when they were received or theirquantity was only partially satisfied, referred to as a partial fill,are maintained by the electronic trading system in an order bookdatabase/data structure to await the subsequent arrival of matchingorders or the occurrence of other conditions which may cause the orderto be modified or otherwise removed from the order book.

If the match engine identifies one or more suitable previously receivedbut unsatisfied counter orders, they, and the incoming order, arematched to execute a trade there between to at least partially satisfythe quantities of one or both the incoming order or the identifiedorders. If there remains any residual unsatisfied quantity of theidentified one or more orders, those orders are left on the order bookwith their remaining quantity to await a subsequent suitable counterorder, i.e., to rest. If the match engine does not identify a suitablepreviously received but unsatisfied counter order, or the one or moreidentified suitable previously received but unsatisfied counter ordersare for a lesser quantity than the incoming order, the incoming order isplaced on the order book, referred to as “resting”, with original orremaining unsatisfied quantity, to await a subsequently receivedsuitable order counter thereto. The match engine then generates matchevent data reflecting the result of this matching process. Othercomponents of the electronic trading system, as will be described, thengenerate the respective order acknowledgment and market data messagesand transmit those messages to the market participants.

Matching, which is a function typically performed by the exchange, is aprocess, for a given order which specifies a desire to buy or sell aquantity of a particular instrument at a particular price, ofseeking/identifying one or more wholly or partially, with respect toquantity, satisfying counter orders thereto, e.g., a sell counter to anorder to buy, or vice versa, for the same instrument at the same, orsometimes better, price (but not necessarily the same quantity), whichare then paired for execution to complete a trade between the respectivemarket participants (via the exchange) and at least partially satisfythe desired quantity of one or both of the order and/or the counterorder, with any residual unsatisfied quantity left to await anothersuitable counter order, referred to as “resting.” A match event mayoccur, for example, when an aggressing order matches with a restingorder. In one embodiment, two orders match because one order includesinstructions for or specifies buying a quantity of a particularinstrument at a particular price, and the other order includesinstructions for or specifies selling a (different or same) quantity ofthe instrument at a same or better price. It should be appreciated thatperforming an instruction associated with a message may includeattempting to perform the instruction. Whether or not an exchangecomputing system is able to successfully perform an instruction maydepend on the state of the electronic marketplace.

While the disclosed embodiments will be described with respect to aproduct by product or market by market implementation, e.g. implementedfor each market/order book, it will be appreciated that the disclosedembodiments may be implemented so as to apply across markets formultiple products traded on one or more electronic trading systems, suchas by monitoring an aggregate, correlated or other derivation of therelevant indicative parameters as described herein.

While the disclosed embodiments may be discussed in relation to futuresand/or options on futures trading, it should be appreciated that thedisclosed embodiments may be applicable to any equity, fixed incomesecurity, currency, commodity, options or futures trading system ormarket now available or later developed. It may be appreciated that atrading environment, such as a futures exchange as described herein,implements one or more economic markets where rights and obligations maybe traded. As such, a trading environment may be characterized by a needto maintain market integrity, transparency, predictability,fair/equitable access and participant expectations with respect thereto.In addition, it may be appreciated that electronic trading systemsfurther impose additional expectations and demands by marketparticipants as to transaction processing speed, latency, capacity andresponse time, while creating additional complexities relating thereto.Accordingly, as will be described, the disclosed embodiments may furtherinclude functionality to ensure that the expectations of marketparticipants are met, e.g., that transactional integrity and predictablesystem responses are maintained.

Financial instrument trading systems allow traders to submit orders andreceive confirmations, market data, and other information electronicallyvia electronic messages exchanged using a network. Electronic tradingsystems ideally attempt to offer a more efficient, fair and balancedmarket where market prices reflect a true consensus of the value oftraded products among the market participants, where the intentional orunintentional influence of any one market participant is minimized ifnot eliminated, and where unfair or inequitable advantages with respectto information access are minimized if not eliminated.

Electronic marketplaces attempt to achieve these goals by usingelectronic messages to communicate actions and related data of theelectronic marketplace between market participants, clearing firms,clearing houses, and other parties. The messages can be received usingan electronic trading system, wherein an action or transactionassociated with the messages may be executed. For example, the messagemay contain information relating to an order to buy or sell a product ina particular electronic marketplace, and the action associated with themessage may indicate that the order is to be placed in the electronicmarketplace such that other orders which were previously placed maypotentially be matched to the order of the received message. Thus, theelectronic marketplace may conduct market activities through electronicsystems.

Clearing House

The clearing house of an exchange clears, settles and guarantees matchedtransactions in contracts occurring through the facilities of theexchange. In addition, the clearing house establishes and monitorsfinancial requirements for clearing members and conveys certain clearingprivileges in conjunction with the relevant exchange markets.

The clearing house establishes clearing level performance bonds(margins) for all products of the exchange and establishes minimumperformance bond requirements for customers of such products. Aperformance bond, also referred to as a margin requirement, correspondswith the funds that must be deposited by a customer with his or herbroker, by a broker with a clearing member or by a clearing member withthe clearing house, for the purpose of insuring the broker or clearinghouse against loss on open futures or options contracts. This is not apart payment on a purchase. The performance bond helps to ensure thefinancial integrity of brokers, clearing members and the exchange as awhole. The performance bond refers to the minimum dollar depositrequired by the clearing house from clearing members in accordance withtheir positions. Maintenance, or maintenance margin, refers to a sum,usually smaller than the initial performance bond, which must remain ondeposit in the customer's account for any position at all times. Theinitial margin is the total amount of margin per contract required bythe broker when a futures position is opened. A drop in funds below thislevel requires a deposit back to the initial margin levels, i.e., aperformance bond call. If a customer's equity in any futures positiondrops to or under the maintenance level because of adverse price action,the broker must issue a performance bond/margin call to restore thecustomer's equity. A performance bond call, also referred to as a margincall, is a demand for additional funds to bring the customer's accountback up to the initial performance bond level whenever adverse pricemovements cause the account to go below the maintenance.

The exchange derives its financial stability in large part by removingdebt obligations among market participants as they occur. This isaccomplished by determining a settlement price at the close of themarket each day for each contract and marking all open positions to thatprice, referred to as “mark to market.” Every contract is debited orcredited based on that trading session's gains or losses. As prices movefor or against a position, funds flow into and out of the tradingaccount. In the case of the CME, each business day by 6:40 a.m. Chicagotime, based on the mark-to-the-market of all open positions to theprevious trading day's settlement price, the clearing house pays to orcollects cash from each clearing member. This cash flow, known assettlement variation, is performed by CME's settlement banks based oninstructions issued by the clearing house. All payments to andcollections from clearing members are made in “same-day” funds. Inaddition to the 6:40 a.m. settlement, a daily intra-day mark-to-themarket of all open positions, including trades executed during theovernight GLOBEX®, the CME's electronic trading systems, trading sessionand the current day's trades matched before 11:15 a.m., is performedusing current prices. The resulting cash payments are made intra-day forsame day value. In times of extreme price volatility, the clearing househas the authority to perform additional intra-day mark-to-the-marketcalculations on open positions and to call for immediate payment ofsettlement variation. CME's mark-to-the-market settlement system differsfrom the settlement systems implemented by many other financial markets,including the interbank, Treasury securities, over-the-counter foreignexchange and debt, options, and equities markets, where participantsregularly assume credit exposure to each other. In those markets, thefailure of one participant can have a ripple effect on the solvency ofthe other participants. Conversely, CME's mark-to-the-market system doesnot allow losses to accumulate over time or allow a market participantthe opportunity to defer losses associated with market positions.

While the disclosed embodiments may be described in reference to theCME, it should be appreciated that these embodiments are applicable toany exchange. Such other exchanges may include a clearing house that,like the CME clearing house, clears, settles and guarantees all matchedtransactions in contracts of the exchange occurring through itsfacilities. In addition, such clearing houses establish and monitorfinancial requirements for clearing members and convey certain clearingprivileges in conjunction with the relevant exchange markets.

The disclosed embodiments are also not limited to uses by a clearinghouse or exchange for purposes of enforcing a performance bond or marginrequirement. For example, a market participant may use the disclosedembodiments in a simulation or other analysis of a portfolio. In suchcases, the settlement price may be useful as an indication of a value atrisk and/or cash flow obligation rather than a performance bond. Thedisclosed embodiments may also be used by market participants or otherentities to forecast or predict the effects of a prospective position onthe margin requirement of the market participant.

Trading Environment

The embodiments may be described in terms of a distributed computingsystem. The particular examples identify a specific set of componentsuseful in a futures and options exchange. However, many of thecomponents and inventive features are readily adapted to otherelectronic trading environments. The specific examples described hereinmay teach specific protocols and/or interfaces, although it should beunderstood that the principals involved may be extended to, or appliedin, other protocols and interfaces.

It should be appreciated that the plurality of entities utilizing orinvolved with the disclosed embodiments, e.g., the market participants,may be referred to by other nomenclature reflecting the role that theparticular entity is performing with respect to the disclosedembodiments and that a given entity may perform more than one roledepending upon the implementation and the nature of the particulartransaction being undertaken, as well as the entity's contractual and/orlegal relationship with another market participant and/or the exchange.

An exemplary trading network environment for implementing tradingsystems and methods is shown in FIG. 1 . An exchange computer system 100receives messages that include orders and transmits market data relatedto orders and trades to users, such as via wide area network 162 and/orlocal area network 160 and computer devices 150, 152, 154, 156 and 158,as described herein, coupled with the exchange computer system 100.

Herein, the phrase “coupled with” is defined to mean directly connectedto or indirectly connected through one or more intermediate components.Such intermediate components may include both hardware and softwarebased components. Further, to clarify the use in the pending claims andto hereby provide notice to the public, the phrases “at least one of<A>, <B>, . . . and <N>” or “at least one of <A>, <B>, . . . <N>, orcombinations thereof” are defined by the Applicant in the broadestsense, superseding any other implied definitions herebefore orhereinafter unless expressly asserted by the Applicant to the contrary,to mean one or more elements selected from the group comprising A, B, .. . and N, that is to say, any combination of one or more of theelements A, B, . . . or N including any one element alone or incombination with one or more of the other elements which may alsoinclude, in combination, additional elements not listed.

The exchange computer system 100 may be implemented with one or moremainframe, desktop or other computers, such as the example computer 200described herein with respect to FIG. 2 . A user database 102 may beprovided which includes information identifying traders and other usersof exchange computer system 100, such as account numbers or identifiers,user names and passwords. An account data module 104 may be providedwhich may process account information that may be used during trades.

A match engine module 106 may be included to match bid and offer pricesand may be implemented with software that executes one or morealgorithms for matching bids and offers. A trade database 108 may beincluded to store information identifying trades and descriptions oftrades. In particular, a trade database may store informationidentifying the time that a trade took place and the contract price. Anorder book module 110 may be included to compute or otherwise determinecurrent bid and offer prices, e.g., in a continuous auction market, oralso operate as an order accumulation buffer for a batch auction market.

A market data module 112 may be included to collect market data andprepare the data for transmission to users.

A risk management module 114 may be included to compute and determine auser's risk utilization in relation to the user's defined riskthresholds. The risk management module 114 may also be configured todetermine risk assessments or exposure levels in connection withpositions held by a market participant. The risk management module 114may be configured to administer, manage or maintain one or moremargining mechanisms implemented by the exchange computer system 100.Such administration, management or maintenance may include managing anumber of database records reflective of margin accounts of the marketparticipants. In some embodiments, the risk management module 114implements one or more aspects of the disclosed embodiments, including,for instance, principal component analysis (PCA) based margining, inconnection with interest rate swap (IRS) portfolios, as describedherein.

A message management module 116 may be included to, among other things,receive, and extract orders from, electronic data transaction requestmessages. The message management module 116 may define a point ofingress into the exchange computer system 100 where messages are orderedand considered to be received by the system. This may be considered apoint of determinism in the exchange computer system 100 that definesthe earliest point where the system can ascribe an order of receipt toarriving messages. The point of determinism may or may not be at or nearthe demarcation point between the exchange computer system 100 and apublic/internet network infrastructure. The message management module116 processes messages by interpreting the contents of a message basedon the message transmit protocol, such as the transmission controlprotocol (“TCP”), to provide the content of the message for furtherprocessing by the exchange computer system.

The message management module 116 may also be configured to detectcharacteristics of an order for a transaction to be undertaken in anelectronic marketplace. For example, the message management module 116may identify and extract order content such as a price, product, volume,and associated market participant for an order. The message managementmodule 116 may also identify and extract data indicating an action to beexecuted by the exchange computer system 100 with respect to theextracted order. For example, the message management module 116 maydetermine the transaction type of the transaction requested in a givenmessage. A message may include an instruction to perform a type oftransaction. The transaction type may be, in one embodiment, arequest/offer/order to either buy or sell a specified quantity or unitsof a financial instrument at a specified price or value. The messagemanagement module 116 may also identify and extract other orderinformation and other actions associated with the extracted order. Allextracted order characteristics, other information, and associatedactions extracted from a message for an order may be collectivelyconsidered an order as described and referenced herein.

Order or message characteristics may include, for example, the state ofthe system after a message is received, arrival time (e.g., the time amessage arrives at the MSG or Market Segment Gateway), message type(e.g., new, modify, cancel), and the number of matches generated by amessage. Order or message characteristics may also include marketparticipant side (e.g., buyer or seller) or time in force (e.g., a gooduntil end of day order that is good for the full trading day, a gooduntil canceled ordered that rests on the order book until matched, or afill or kill order that is canceled if not filled immediately, or a filland kill order (FOK) that is filled to the maximum amount possible, andany remaining or unfilled/unsatisfied quantity is not stored on thebooks or allowed to rest).

An order processing module 118 may be included to decompose delta-based,spread instrument, bulk and other types of composite orders forprocessing by the order book module 110 and/or the match engine module106. The order processing module 118 may also be used to implement oneor more procedures related to clearing an order. The order may becommunicated from the message management module 118 to the orderprocessing module 118. The order processing module 118 may be configuredto interpret the communicated order, and manage the ordercharacteristics, other information, and associated actions as they areprocessed through an order book module 110 and eventually transacted onan electronic market. For example, the order processing module 118 maystore the order characteristics and other content and execute theassociated actions. In an embodiment, the order processing module mayexecute an associated action of placing the order into an order book foran electronic trading system managed by the order book module 110. In anembodiment, placing an order into an order book and/or into anelectronic trading system may be considered a primary action for anorder. The order processing module 118 may be configured in variousarrangements, and may be configured as part of the order book module110, part of the message management module 118, or as an independentfunctioning module.

As an intermediary to electronic trading transactions, the exchangebears a certain amount of risk in each transaction that takes place. Tothat end, the clearing house implements risk management mechanisms toprotect the exchange. One or more of the modules of the exchangecomputer system 100 may be configured to determine settlement prices forconstituent contracts, such as deferred month contracts, of spreadinstruments, such as for example, settlement module 120. A settlementmodule 120 (or settlement processor or other payment processor) may beincluded to provide one or more functions related to settling orotherwise administering transactions cleared by the exchange. Settlementmodule 120 of the exchange computer system 100 may implement one or moresettlement price determination techniques. Settlement-related functionsneed not be limited to actions or events occurring at the end of acontract term. For instance, in some embodiments, settlement-relatedfunctions may include or involve daily or other mark to marketsettlements for margining purposes. In some cases, the settlement module120 may be configured to communicate with the trade database 108 (or thememory(ies) on which the trade database 108 is stored) and/or todetermine a payment amount based on a spot price, the price of thefutures contract or other financial instrument, or other price data, atvarious times. The determination may be made at one or more points intime during the term of the financial instrument in connection with amargining mechanism. For example, the settlement module 120 may be usedto determine a mark to market amount on a daily basis during the term ofthe financial instrument. Such determinations may also be made on asettlement date for the financial instrument for the purposes of finalsettlement.

In some embodiments, the settlement module 120 may be integrated to anydesired extent with one or more of the other modules or processors ofthe exchange computer system 100. For example, the settlement module 120and the risk management module 114 may be integrated to any desiredextent. In some cases, one or more margining procedures or other aspectsof the margining mechanism(s) may be implemented by the settlementmodule 120.

One or more of the above-described modules of the exchange computersystem 100 may be used to gather or obtain data to support thesettlement price determination, as well as a subsequent marginrequirement determination. For example, the order book module 110 and/orthe market data module 112 may be used to receive, access, or otherwiseobtain market data, such as bid-offer values of orders currently on theorder books. The trade database 108 may be used to receive, access, orotherwise obtain trade data indicative of the prices and volumes oftrades that were recently executed in a number of markets. In somecases, transaction data (and/or bid/ask data) may be gathered orobtained from open outcry pits and/or other sources and incorporatedinto the trade and market data from the electronic trading system(s).

It should be appreciated that concurrent processing limits may bedefined by or imposed separately or in combination on one or more of thetrading system components, including the user database 102, the accountdata module 104, the match engine module 106, the trade database 108,the order book module 110, the market data module 112, the riskmanagement module 114, the message management module 116, the orderprocessing module 118, the settlement module 120, or other component ofthe exchange computer system 100.

The disclosed mechanisms may be implemented at any logical and/orphysical point(s), or combinations thereof, at which the relevantinformation/data (e.g., message traffic and responses thereto) may bemonitored or flows or is otherwise accessible or measurable, includingone or more gateway devices, modems, the computers or terminals of oneor more market participants, e.g., client computers, etc.

One skilled in the art will appreciate that one or more modulesdescribed herein may be implemented using, among other things, atangible computer-readable medium comprising computer-executableinstructions (e.g., executable software code). Alternatively, modulesmay be implemented as software code, firmware code, specificallyconfigured hardware or processors, and/or a combination of theaforementioned. For example, the modules may be embodied as part of anexchange 100 for financial instruments. It should be appreciated thedisclosed embodiments may be implemented as a different or separatemodule of the exchange computer system 100, or a separate computersystem coupled with the exchange computer system 100 so as to haveaccess to margin account record, pricing, and/or other data. Asdescribed herein, the disclosed embodiments may be implemented as acentrally accessible system or as a distributed system, e.g., where someof the disclosed functions are performed by the computer systems of themarket participants.

The trading network environment shown in FIG. 1 includes exemplarycomputer devices 150, 152, 154, 156 and 158 which depict differentexemplary methods or media by which a computer device may be coupledwith the exchange computer system 100 or by which a user maycommunicate, e.g., send and receive, trade or other informationtherewith. It should be appreciated that the types of computer devicesdeployed by traders and the methods and media by which they communicatewith the exchange computer system 100 is implementation dependent andmay vary and that not all of the depicted computer devices and/ormeans/media of communication may be used and that other computer devicesand/or means/media of communications, now available or later developedmay be used. Each computer device, which may comprise a computer 200described in more detail with respect to FIG. 2 , may include a centralprocessor, specifically configured or otherwise, that controls theoverall operation of the computer and a system bus that connects thecentral processor to one or more conventional components, such as anetwork card or modem. Each computer device may also include a varietyof interface units and drives for reading and writing data or files andcommunicating with other computer devices and with the exchange computersystem 100. Depending on the type of computer device, a user caninteract with the computer with a keyboard, pointing device, microphone,pen device or other input device now available or later developed.

An exemplary computer device 150 is shown directly connected to exchangecomputer system 100, such as via a T1 line, a common local area network(LAN) or other wired and/or wireless medium for connecting computerdevices, such as the network 220 shown in FIG. 2 and described withrespect thereto. The exemplary computer device 150 is further shownconnected to a radio 168. The user of radio 168, which may include acellular telephone, smart phone, or other wireless proprietary and/ornon-proprietary device, may be a trader or exchange employee. The radiouser may transmit orders or other information to the exemplary computerdevice 150 or a user thereof. The user of the exemplary computer device150, or the exemplary computer device 150 alone and/or autonomously, maythen transmit the trade or other information to the exchange computersystem 100.

Exemplary computer devices 152 and 154 are coupled with a local areanetwork (“LAN”) 160 which may be configured in one or more of thewell-known LAN topologies, e.g., star, daisy chain, etc., and may use avariety of different protocols, such as Ethernet, TCP/IP, etc. Theexemplary computer devices 152 and 154 may communicate with each otherand with other computers and other devices which are coupled with theLAN 160. Computer and other devices may be coupled with the LAN 160 viatwisted pair wires, coaxial cable, fiber optics or other wired orwireless media. As shown in FIG. 1 , an exemplary wireless personaldigital assistant device (“PDA”) 158, such as a mobile telephone, tabletbased compute device, or other wireless device, may communicate with theLAN 160 and/or the Internet 162 via radio waves, such as via WiFi,Bluetooth and/or a cellular telephone based data communicationsprotocol. PDA 158 may also communicate with exchange computer system 100via a conventional wireless hub 164.

FIG. 1 also shows the LAN 160 coupled with a wide area network (“WAN”)162 which may be comprised of one or more public or private wired orwireless networks. In one embodiment, the WAN 162 includes the Internet162. The LAN 160 may include a router to connect LAN 160 to the Internet162. Exemplary computer device 156 is shown coupled directly to theInternet 162, such as via a modem, DSL line, satellite dish or any otherdevice for connecting a computer device to the Internet 162 via aservice provider therefore as is known. LAN 160 and/or WAN 162 may bethe same as the network 220 shown in FIG. 2 and described with respectthereto.

Users of the exchange computer system 100 may include one or more marketmakers 166 which may maintain a market by providing constant bid andoffer prices for a derivative or security to the exchange computersystem 100, such as via one of the exemplary computer devices depicted.The exchange computer system 100 may also exchange information withother match or trade engines, such as trade engine 170. One skilled inthe art will appreciate that numerous additional computers and systemsmay be coupled to exchange computer system 100. Such computers andsystems may include clearing, regulatory and fee systems.

The operations of computer devices and systems shown in FIG. 1 may becontrolled by computer-executable instructions stored on anon-transitory computer-readable medium. For example, the exemplarycomputer device 152 may store computer-executable instructions forreceiving order information from a user, transmitting that orderinformation to exchange computer system 100 in electronic messages,extracting the order information from the electronic messages, executingactions relating to the messages, and/or calculating values fromcharacteristics of the extracted order to facilitate matching orders andexecuting trades. In another example, the exemplary computer device 154may include computer-executable instructions for receiving market datafrom exchange computer system 100 and displaying that information to auser.

Numerous additional servers, computers, handheld devices, personaldigital assistants, telephones and other devices may also be connectedto exchange computer system 100. Moreover, one skilled in the art willappreciate that the topology shown in FIG. 1 is merely an example andthat the components shown in FIG. 1 may include other components notshown and be connected by numerous alternative topologies.

Referring now to FIG. 2 , an illustrative embodiment of a generalcomputer system 200 is shown. The computer system 200 can include a setof instructions that can be executed to cause the computer system 200 toperform any one or more of the methods or computer based functionsdisclosed herein. The computer system 200 may operate as a standalonedevice or may be connected, e.g., using a network, to other computersystems or peripheral devices. Any of the components discussed herein,such as processor 202, may be a computer system 200 or a component inthe computer system 200. The computer system 200 may be specificallyconfigured to implement a match engine, margin processing, payment orclearing function on behalf of an exchange, such as the ChicagoMercantile Exchange, of which the disclosed embodiments are a componentthereof.

In a networked deployment, the computer system 200 may operate in thecapacity of a server or as a client user computer in a client-serveruser network environment, or as a peer computer system in a peer-to-peer(or distributed) network environment. The computer system 200 can alsobe implemented as or incorporated into various devices, such as apersonal computer (PC), a tablet PC, a set-top box (STB), a personaldigital assistant (PDA), a mobile device, a palmtop computer, a laptopcomputer, a desktop computer, a communications device, a wirelesstelephone, a land-line telephone, a control system, a camera, a scanner,a facsimile machine, a printer, a pager, a personal trusted device, aweb appliance, a network router, switch or bridge, or any other machinecapable of executing a set of instructions (sequential or otherwise)that specify actions to be taken by that machine. In a particularembodiment, the computer system 200 can be implemented using electronicdevices that provide voice, video or data communication. Further, whilea single computer system 200 is illustrated, the term “system” shallalso be taken to include any collection of systems or sub-systems thatindividually or jointly execute a set, or multiple sets, of instructionsto perform one or more computer functions.

As illustrated in FIG. 2 , the computer system 200 may include aprocessor 202, e.g., a central processing unit (CPU), a graphicsprocessing unit (GPU), or both. The processor 202 may be a component ina variety of systems. For example, the processor 202 may be part of astandard personal computer or a workstation. The processor 202 may beone or more general processors, digital signal processors, specificallyconfigured processors, application specific integrated circuits, fieldprogrammable gate arrays, servers, networks, digital circuits, analogcircuits, combinations thereof, or other now known or later developeddevices for analyzing and processing data. The processor 202 mayimplement a software program, such as code generated manually (i.e.,programmed).

The computer system 200 may include a memory 204 that can communicatevia a bus 208. The memory 204 may be a main memory, a static memory, ora dynamic memory. The memory 204 may include, but is not limited to,computer readable storage media such as various types of volatile andnon-volatile storage media, including but not limited to random accessmemory, read-only memory, programmable read-only memory, electricallyprogrammable read-only memory, electrically erasable read-only memory,flash memory, magnetic tape or disk, optical media and the like. In oneembodiment, the memory 204 includes a cache or random access memory forthe processor 202. In alternative embodiments, the memory 204 isseparate from the processor 202, such as a cache memory of a processor,the system memory, or other memory. The memory 204 may be an externalstorage device or database for storing data. Examples include a harddrive, compact disc (“CD”), digital video disc (“DVD”), memory card,memory stick, floppy disc, universal serial bus (“USB”) memory device,or any other device operative to store data. The memory 204 is operableto store instructions executable by the processor 202. The functions,acts or tasks illustrated in the figures or described herein may beperformed by the programmed processor 202 executing the instructions 212stored in the memory 204. The functions, acts or tasks are independentof the particular type of instructions set, storage media, processor orprocessing strategy and may be performed by software, hardware,integrated circuits, firm-ware, micro-code and the like, operating aloneor in combination. Likewise, processing strategies may includemultiprocessing, multitasking, parallel processing and the like.

As shown, the computer system 200 may further include a display unit214, such as a liquid crystal display (LCD), an organic light emittingdiode (OLED), a flat panel display, a solid state display, a cathode raytube (CRT), a projector, a printer or other now known or later developeddisplay device for outputting determined information. The display 214may act as an interface for the user to see the functioning of theprocessor 202, or specifically as an interface with the software storedin the memory 204 or in the drive unit 206.

Additionally, the computer system 200 may include an input device 216configured to allow a user to interact with any of the components ofsystem 200. The input device 216 may be a number pad, a keyboard, or acursor control device, such as a mouse, or a joystick, touch screendisplay, remote control or any other device operative to interact withthe system 200.

In a particular embodiment, as depicted in FIG. 2 , the computer system200 may also include a disk or optical drive unit 206. The disk driveunit 206 may include a computer-readable medium 210 in which one or moresets of instructions 212, e.g., software, can be embedded. Further, theinstructions 212 may embody one or more of the methods or logic asdescribed herein. In a particular embodiment, the instructions 212 mayreside completely, or at least partially, within the memory 204 and/orwithin the processor 202 during execution by the computer system 200.The memory 204 and the processor 202 also may include computer-readablemedia as discussed herein.

The present disclosure contemplates a computer-readable medium thatincludes instructions 212 or receives and executes instructions 212responsive to a propagated signal, so that a device connected to anetwork 220 can communicate voice, video, audio, images or any otherdata over the network 220. Further, the instructions 212 may betransmitted or received over the network 220 via a communicationinterface 218. The communication interface 218 may be a part of theprocessor 202 or may be a separate component. The communicationinterface 218 may be created in software or may be a physical connectionin hardware. The communication interface 218 is configured to connectwith a network 220, external media, the display 214, or any othercomponents in system 200, or combinations thereof. The connection withthe network 220 may be a physical connection, such as a wired Ethernetconnection or may be established wirelessly. Likewise, the additionalconnections with other components of the system 200 may be physicalconnections or may be established wirelessly.

The network 220 may include wired networks, wireless networks, orcombinations thereof. The wireless network may be a cellular telephonenetwork, an 802.11, 802.16, 802.20, or WiMax network. Further, thenetwork 220 may be a public network, such as the Internet, a privatenetwork, such as an intranet, or combinations thereof, and may utilize avariety of networking protocols now available or later developedincluding, but not limited to, TCP/IP based networking protocols.

Embodiments of the subject matter and the functional operationsdescribed in this specification can be implemented in digital electroniccircuitry, or in computer software, firmware, or hardware, including thestructures disclosed in this specification and their structuralequivalents, or in combinations of one or more of them. Embodiments ofthe subject matter described in this specification can be implemented asone or more computer program products, i.e., one or more modules ofcomputer program instructions encoded on a computer readable medium forexecution by, or to control the operation of, data processing apparatus.While the computer-readable medium is shown to be a single medium, theterm “computer-readable medium” includes a single medium or multiplemedia, such as a centralized or distributed database, and/or associatedcaches and servers that store one or more sets of instructions. The term“computer-readable medium” shall also include any medium that is capableof storing, encoding or carrying a set of instructions for execution bya processor or that cause a computer system to perform any one or moreof the methods or operations disclosed herein. The computer readablemedium can be a machine-readable storage device, a machine-readablestorage substrate, a memory device, or a combination of one or more ofthem. The term “data processing apparatus” encompasses all apparatus,devices, and machines for processing data, including by way of example aprogrammable processor, a computer, or multiple processors or computers.The apparatus can include, in addition to hardware, code that creates anexecution environment for the computer program in question, e.g., codethat constitutes processor firmware, a protocol stack, a databasemanagement system, an operating system, or a combination of one or moreof them.

In a particular non-limiting, exemplary embodiment, thecomputer-readable medium can include a solid-state memory such as amemory card or other package that houses one or more non-volatileread-only memories. Further, the computer-readable medium can be arandom access memory or other volatile re-writable memory. Additionally,the computer-readable medium can include a magneto-optical or opticalmedium, such as a disk or tapes or other storage device to capturecarrier wave signals such as a signal communicated over a transmissionmedium. A digital file attachment to an e-mail or other self-containedinformation archive or set of archives may be considered a distributionmedium that is a tangible storage medium. Accordingly, the disclosure isconsidered to include any one or more of a computer-readable medium or adistribution medium and other equivalents and successor media, in whichdata or instructions may be stored.

In an alternative embodiment, dedicated or otherwise specificallyconfigured hardware implementations, such as application specificintegrated circuits, programmable logic arrays and other hardwaredevices, can be constructed to implement one or more of the methodsdescribed herein. Applications that may include the apparatus andsystems of various embodiments can broadly include a variety ofelectronic and computer systems. One or more embodiments describedherein may implement functions using two or more specific interconnectedhardware modules or devices with related control and data signals thatcan be communicated between and through the modules, or as portions ofan application-specific integrated circuit. Accordingly, the presentsystem encompasses software, firmware, and hardware implementations.

In accordance with various embodiments of the present disclosure, themethods described herein may be implemented by software programsexecutable by a computer system. Further, in an exemplary, non-limitedembodiment, implementations can include distributed processing,component/object distributed processing, and parallel processing.Alternatively, virtual computer system processing can be constructed toimplement one or more of the methods or functionality as describedherein.

Although the present specification describes components and functionsthat may be implemented in particular embodiments with reference toparticular standards and protocols, the invention is not limited to suchstandards and protocols. For example, standards for Internet and otherpacket switched network transmission (e.g., TCP/IP, UDP/IP, HTML, HTTP,HTTPS) represent examples of the state of the art. Such standards areperiodically superseded by faster or more efficient equivalents havingessentially the same functions. Accordingly, replacement standards andprotocols having the same or similar functions as those disclosed hereinare considered equivalents thereof.

A computer program (also known as a program, software, softwareapplication, script, or code) can be written in any form of programminglanguage, including compiled or interpreted languages, and it can bedeployed in any form, including as a standalone program or as a module,component, subroutine, or other unit suitable for use in a computingenvironment. A computer program does not necessarily correspond to afile in a file system. A program can be stored in a portion of a filethat holds other programs or data (e.g., one or more scripts stored in amarkup language document), in a single file dedicated to the program inquestion, or in multiple coordinated files (e.g., files that store oneor more modules, sub programs, or portions of code). A computer programcan be deployed to be executed on one computer or on multiple computersthat are located at one site or distributed across multiple sites andinterconnected by a communication network.

The processes and logic flows described in this specification can beperformed by one or more programmable processors executing one or morecomputer programs to perform functions by operating on input data andgenerating output. The processes and logic flows can also be performedby, and apparatus can also be implemented as, special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) or an ASIC(application specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andanyone or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read only memory ora random access memory or both. The essential elements of a computer area processor for performing instructions and one or more memory devicesfor storing instructions and data. Generally, a computer will alsoinclude, or be operatively coupled to receive data from or transfer datato, or both, one or more mass storage devices for storing data, e.g.,magnetic, magneto optical disks, or optical disks. However, a computerneed not have such devices. Moreover, a computer can be embedded inanother device, e.g., a mobile telephone, a personal digital assistant(PDA), a mobile audio player, a Global Positioning System (GPS)receiver, to name just a few. Computer readable media suitable forstoring computer program instructions and data include all forms ofnon-volatile memory, media and memory devices, including by way ofexample semiconductor memory devices, e.g., EPROM, EEPROM, and flashmemory devices; magnetic disks, e.g., internal hard disks or removabledisks; magneto optical disks; and CD ROM and DVD-ROM disks. Theprocessor and the memory can be supplemented by, or incorporated in,special purpose logic circuitry.

As used herein, the terms “microprocessor” or “general-purposeprocessor” (“GPP”) may refer to a hardware device that fetchesinstructions and data from a memory or storage device and executes thoseinstructions (for example, an Intel Xeon processor or an AMD Opteronprocessor) to then, for example, process the data in accordancetherewith. The term “reconfigurable logic” may refer to any logictechnology whose form and function can be significantly altered (i.e.,reconfigured) in the field post-manufacture as opposed to amicroprocessor, whose function can change post-manufacture, e.g. viacomputer executable software code, but whose form, e.g. thearrangement/layout and interconnection of logical structures, is fixedat manufacture. The term “software” may refer to data processingfunctionality that is deployed on a GPP. The term “firmware” may referto data processing functionality that is deployed on reconfigurablelogic. One example of a reconfigurable logic is a field programmablegate array (“FPGA”) which is a reconfigurable integrated circuit. AnFPGA may contain programmable logic components called “logic blocks”,and a hierarchy of reconfigurable interconnects that allow the blocks tobe “wired together”, somewhat like many (changeable) logic gates thatcan be inter-wired in (many) different configurations. Logic blocks maybe configured to perform complex combinatorial functions, or merelysimple logic gates like AND, OR, NOT and XOR. An FPGA may furtherinclude memory elements, which may be simple flip-flops or more completeblocks of memory.

To provide for interaction with a user, embodiments of the subjectmatter described in this specification can be implemented on a devicehaving a display, e.g., a CRT (cathode ray tube) or LCD (liquid crystaldisplay) monitor, for displaying information to the user and a keyboardand a pointing device, e.g., a mouse or a trackball, by which the usercan provide input to the computer. Other kinds of devices can be used toprovide for interaction with a user as well. Feedback provided to theuser can be any form of sensory feedback, e.g., visual feedback,auditory feedback, or tactile feedback. Input from the user can bereceived in any form, including acoustic, speech, or tactile input.

Embodiments of the subject matter described in this specification can beimplemented in a computing system that includes a back end component,e.g., a data server, or that includes a middleware component, e.g., anapplication server, or that includes a front end component, e.g., aclient computer having a graphical user interface or a Web browserthrough which a user can interact with an implementation of the subjectmatter described in this specification, or any combination of one ormore such back end, middleware, or front end components. The componentsof the system can be interconnected by any form or medium of digitaldata communication, e.g., a communication network. Examples ofcommunication networks include a local area network (“LAN”) and a widearea network (“WAN”), e.g., the Internet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

It should be appreciated that the disclosed embodiments may beapplicable to other types of messages depending upon the implementation.Further, the messages may comprise one or more data packets, datagramsor other collection of data formatted, arranged configured and/orpackaged in a particular one or more protocols, e.g., the FIX protocol,TCP/IP, Ethernet, etc., suitable for transmission via a network 214 aswas described, such as the message format and/or protocols described inU.S. Pat. No. 7,831,491 and U.S. Patent Publication No. 2005/0096999 A1,both of which are incorporated by reference herein in their entiretiesand relied upon. Further, the disclosed message management system may beimplemented using an open message standard implementation, such as FIX,FIX Binary, FIX/FAST, or by an exchange-provided API.

The embodiments described herein utilize trade related electronicmessages such as mass quote messages, individual order messages,modification messages, cancellation messages, etc., so as to enacttrading activity in an electronic market. The trading entity and/ormarket participant may have one or multiple trading terminals associatedwith the session. Furthermore, the financial instruments may befinancial derivative products. Derivative products may include futurescontracts, options on futures contracts, futures contracts that arefunctions of or related to other futures contracts, swaps, swaptions, orother financial instruments that have their price related to or derivedfrom an underlying product, security, commodity, equity, index, orinterest rate product. In one embodiment, the orders are for optionscontracts that belong to a common option class. Orders may also be forbaskets, quadrants, other combinations of financial instruments, etc.The option contracts may have a plurality of strike prices and/orcomprise put and call contracts. A mass quote message may be received atan exchange. As used herein, an exchange computing system 100 includes aplace or system that receives and/or executes orders.

In an embodiment, a plurality of electronic messages is received fromthe network. The plurality of electronic messages may be received at anetwork interface for the electronic trading system. The plurality ofelectronic messages may be sent from market participants. The pluralityof messages may include order characteristics and be associated withactions to be executed with respect to an order that may be extractedfrom the order characteristics. The action may involve any action asassociated with transacting the order in an electronic trading system.The actions may involve placing the orders within a particular marketand/or order book of a market in the electronic trading system.

In an embodiment, an incoming transaction may be received. The incomingtransaction may be from, and therefore associated with, a marketparticipant of an electronic market managed by an electronic tradingsystem. The transaction may involve an order as extracted from areceived message, and may have an associated action. The actions mayinvolve placing an order to buy or sell a financial product in theelectronic market, or modifying or deleting such an order. In anembodiment, the financial product may be based on an associatedfinancial instrument which the electronic market is established totrade.

In an embodiment, the action associated with the transaction isdetermined. For example, it may be determined whether the incomingtransaction comprises an order to buy or sell a quantity of theassociated financial instrument or an order to modify or cancel anexisting order in the electronic market. Orders to buy or sell andorders to modify or cancel may be acted upon differently by theelectronic market. For example, data indicative of differentcharacteristics of the types of orders may be stored.

In an embodiment, data relating to the received transaction is stored.The data may be stored in any device, or using any technique, operableto store and provide recovery of data. For example, a memory 204 orcomputer readable medium 210, may be used to store data, as is describedwith respect to FIG. 2 in further detail herein. Data may be storedrelating received transactions for a period of time, indefinitely, orfor a rolling most recent time period such that the stored data isindicative of the market participant's recent activity in the electronicmarket.

If and/or when a transaction is determined to be an order to modify orcancel a previously placed, or existing, order, data indicative of theseactions may be stored. For example, data indicative of a running countof a number or frequency of the receipt of modify or cancel orders fromthe market participant may be stored. A number may be a total number ofmodify or cancel orders received from the market participant, or anumber of modify or cancel orders received from the market participantover a specified time. A frequency may be a time based frequency, as ina number of cancel or modify orders per unit of time, or a number ofcancel or modify orders received from the market participant as apercentage of total transactions received from the participant, whichmay or may not be limited by a specified length of time.

If and/or when a transaction is determined to be an order to buy or sella financial product, or financial instrument, other indicative data maybe stored. For example, data indicative of quantity and associated priceof the order to buy or sell may be stored.

Data indicative of attempts to match incoming orders may also be stored.The data may be stored in any device, or using any technique, operableto store and provide recovery of data. For example, a memory 204 orcomputer readable medium 210, may be used to store data, as is describedwith respect to FIG. 2 . The acts of the process as described herein mayalso be repeated. As such, data for multiple received transactions formultiple market participants may be stored and used as described herein.

The order processing module 118 may also store data indicative ofcharacteristics of the extracted orders. For example, the orderprocessing module may store data indicative of orders having anassociated modify or cancel action, such as by recording a count of thenumber of such orders associated with particular market participants.The order processing module may also store data indicative of quantitiesand associated prices of orders to buy or sell a product placed in themarket order book 110, as associated with particular marketparticipants.

Also, the order processing module 118 may be configured to calculate andassociate with particular orders a value indicative of an associatedmarket participant's market activity quality, which is a valueindicative of whether the market participant's market activity increasesor tends to increase liquidity of a market. This value may be determinedbased on the price of the particular order, previously stored quantitiesof orders from the associated market participant, the previously storeddata indicative of previously received orders to modify or cancel asassociated with the market participant, and previously stored dataindicative of a result of the attempt to match previously receivedorders stored in association with the market participant. The orderprocessing module 118 may determine or otherwise calculate scoresindicative of the quality value based on these stored extracted ordercharacteristics, such as an MQI as described herein.

Further, electronic trading systems may perform actions on orders placedfrom received messages based on various characteristics of the messagesand/or market participants associated with the messages. These actionsmay include matching the orders either during a continuous auctionprocess, or at the conclusion of a collection period during a batchauction process. The matching of orders may be by any technique.

The matching of orders may occur based on a priority indicated by thecharacteristics of orders and market participants associated with theorders. Orders having a higher priority may be matched before orders ofa lower priority. Such priority may be determined using varioustechniques. For example, orders that were indicated by messages receivedearlier may receive a higher priority to match than orders that wereindicated by messages received later. Also, scoring or grading of thecharacteristics may provide for priority determination. Data indicativeof order matches may be stored by a match engine and/or an orderprocessing module 118, and used for determining MQI scores of marketparticipants.

Example Users

Generally, a market may involve market makers, such as marketparticipants who consistently provide bids and/or offers at specificprices in a manner typically conducive to balancing risk, and markettakers who may be willing to execute transactions at prevailing bids oroffers may be characterized by more aggressive actions so as to maintainrisk and/or exposure as a speculative investment strategy. From analternate perspective, a market maker may be considered a marketparticipant who places an order to sell at a price at which there is nopreviously or concurrently provided counter order. Similarly, a markettaker may be considered a market participant who places an order to buyat a price at which there is a previously or concurrently providedcounter order. A balanced and efficient market may involve both marketmakers and market takers, coexisting in a mutually beneficial basis. Themutual existence, when functioning properly, may facilitate liquidity inthe market such that a market may exist with “tight” bid-ask spreads(e.g., small difference between bid and ask prices) and a “deep” volumefrom many currently provided orders such that large quantity orders maybe executed without driving prices significantly higher or lower.

As such, both market participant types are useful in generatingliquidity in a market, but specific characteristics of market activitytaken by market participants may provide an indication of a particularmarket participant's effect on market liquidity. For example, a MarketQuality Index (“MQI”) of an order may be determined using thecharacteristics. An MQI may be considered a value indicating alikelihood that a particular order will improve or facilitate liquidityin a market. That is, the value may indicate a likelihood that the orderwill increase a probability that subsequent requests and transactionfrom other market participants will be satisfied. As such, an MQI may bedetermined based on a proximity of the entered price of an order to amidpoint of a current bid-ask price spread, a size of the entered order,a volume or quantity of previously filled orders of the marketparticipant associated with the order, and/or a frequency ofmodifications to previous orders of the market participant associatedwith the order. In this way, an electronic trading system may functionto assess and/or assign an MQI to received electronic messages toestablish messages that have a higher value to the system, and thus thesystem may use computing resources more efficiently by expendingresources to match orders of the higher value messages prior toexpending resources of lower value messages.

While an MQI may be applied to any or all market participants, such anindex may also be applied only to a subset thereof, such as large marketparticipants, or market participants whose market activity as measuredin terms of average daily message traffic over a limited historical timeperiod exceeds a specified number. For example, a market participantgenerating more than 500, 1,000, or even 10,000 market messages per daymay be considered a large market participant.

An exchange provides one or more markets for the purchase and sale ofvarious types of products including financial instruments such asstocks, bonds, futures contracts, options, currency, cash, and othersimilar instruments. Agricultural products and commodities are alsoexamples of products traded on such exchanges. A futures contract is aproduct that is a contract for the future delivery of another financialinstrument such as a quantity of grains, metals, oils, bonds, currency,or cash. Generally, each exchange establishes a specification for eachmarket provided thereby that defines at least the product traded in themarket, minimum quantities that must be traded, and minimum changes inprice (e.g., tick size). For some types of products (e.g., futures oroptions), the specification further defines a quantity of the underlyingproduct represented by one unit (or lot) of the product, and deliveryand expiration dates. As will be described, the exchange may furtherdefine the matching algorithm, or rules, by which incoming orders willbe matched/allocated to resting orders.

Matching and Transaction Processing

Market participants, e.g., traders, use software to send orders ormessages to the trading platform. The order identifies the product, thequantity of the product the trader wishes to trade, a price at which thetrader wishes to trade the product, and a direction of the order (i.e.,whether the order is a bid, i.e., an offer to buy, or an ask, i.e., anoffer to sell). It will be appreciated that there may be other ordertypes or messages that traders can send including requests to modify orcancel a previously submitted order.

The exchange computer system monitors incoming orders received therebyand attempts to identify, i.e., match or allocate, as described herein,one or more previously received, but not yet matched, orders, i.e.,limit orders to buy or sell a given quantity at a given price, referredto as “resting” orders, stored in an order book database, wherein eachidentified order is contra to the incoming order and has a favorableprice relative to the incoming order. An incoming order may be an“aggressor” order, i.e., a market order to sell a given quantity atwhatever may be the current resting bid order price(s) or a market orderto buy a given quantity at whatever may be the current resting ask orderprice(s). An incoming order may be a “market making” order, i.e., amarket order to buy or sell at a price for which there are currently noresting orders. In particular, if the incoming order is a bid, i.e., anoffer to buy, then the identified order(s) will be an ask, i.e., anoffer to sell, at a price that is identical to or higher than the bidprice. Similarly, if the incoming order is an ask, i.e., an offer tosell, the identified order(s) will be a bid, i.e., an offer to buy, at aprice that is identical to or lower than the offer price.

An exchange computing system may receive conditional orders or messagesfor a data object, where the order may include two prices or values: areference value and a stop value. A conditional order may be configuredso that when a product represented by the data object trades at thereference price, the stop order is activated at the stop value. Forexample, if the exchange computing system's order management moduleincludes a stop order with a stop price of 5 and a limit price of 1 fora product, and a trade at 5 (i.e., the stop price of the stop order)occurs, then the exchange computing system attempts to trade at 1 (i.e.,the limit price of the stop order). In other words, a stop order is aconditional order to trade (or execute) at the limit price that istriggered (or elected) when a trade at the stop price occurs.

Stop orders also rest on, or are maintained in, an order book to monitorfor a trade at the stop price, which triggers an attempted trade at thelimit price. In some embodiments, a triggered limit price for a stoporder may be treated as an incoming order.

Upon identification (matching) of a contra order(s), a minimum of thequantities associated with the identified order and the incoming orderis matched and that quantity of each of the identified and incomingorders become two halves of a matched trade that is sent to a clearinghouse. The exchange computer system considers each identified order inthis manner until either all of the identified orders have beenconsidered or all of the quantity associated with the incoming order hasbeen matched, i.e., the order has been filled. If any quantity of theincoming order remains, an entry may be created in the order bookdatabase and information regarding the incoming order is recordedtherein, i.e., a resting order is placed on the order book for theremaining quantity to await a subsequent incoming order counter thereto.

It should be appreciated that in electronic trading systems implementedvia an exchange computing system, a trade price (or match value) maydiffer from (i.e., be better for the submitter, e.g., lower than asubmitted buy price or higher than a submitted sell price) the limitprice that is submitted, e.g., a price included in an incoming message,or a triggered limit price from a stop order.

As used herein, “better” than a reference value means lower than thereference value if the transaction is a purchase (or acquire)transaction, and higher than the reference value if the transaction is asell transaction. Said another way, for purchase (or acquire)transactions, lower values are better, and for relinquish or selltransactions, higher values are better.

Traders access the markets on a trading platform using trading softwarethat receives and displays at least a portion of the order book for amarket, i.e., at least a portion of the currently resting orders,enables a trader to provide parameters for an order for the producttraded in the market, and transmits the order to the exchange computersystem. The trading software typically includes a graphical userinterface to display at least a price and quantity of some of theentries in the order book associated with the market. The number ofentries of the order book displayed is generally preconfigured by thetrading software, limited by the exchange computer system, or customizedby the user. Some graphical user interfaces display order books ofmultiple markets of one or more trading platforms. The trader may be anindividual who trades on his/her behalf, a broker trading on behalf ofanother person or entity, a group, or an entity. Furthermore, the tradermay be a system that automatically generates and submits orders.

If the exchange computer system identifies that an incoming market ordermay be filled by a combination of multiple resting orders, e.g., theresting order at the best price only partially fills the incoming order,the exchange computer system may allocate the remaining quantity of theincoming, i.e., that which was not filled by the resting order at thebest price, among such identified orders in accordance withprioritization and allocation rules/algorithms, referred to as“allocation algorithms” or “matching algorithms,” as, for example, maybe defined in the specification of the particular financial product ordefined by the exchange for multiple financial products. Similarly, ifthe exchange computer system identifies multiple orders contra to theincoming limit order and that have an identical price which is favorableto the price of the incoming order, i.e., the price is equal to orbetter, e.g., lower if the incoming order is a buy (or instruction topurchase, or instruction to acquire) or higher if the incoming order isa sell (or instruction to relinquish), than the price of the incomingorder, the exchange computer system may allocate the quantity of theincoming order among such identified orders in accordance with thematching algorithms as, for example, may be defined in the specificationof the particular financial product or defined by the exchange formultiple financial products.

An exchange responds to inputs, such as trader orders, cancellation,etc., in a manner as expected by the market participants, such as basedon market data, e.g., prices, available counter-orders, etc., to providean expected level of certainty that transactions will occur in aconsistent and predictable manner and without unknown or unascertainablerisks. Accordingly, the method by which incoming orders are matched withresting orders must be defined so that market participants have anexpectation of what the result will be when they place an order or haveresting orders and an incoming order is received, even if the expectedresult is, in fact, at least partially unpredictable due to somecomponent of the process being random or arbitrary or due to marketparticipants having imperfect or less than all information, e.g.,unknown position of an order in an order book. Typically, the exchangedefines the matching/allocation algorithm that will be used for aparticular financial product, with or without input from the marketparticipants. Once defined for a particular product, thematching/allocation algorithm is typically not altered, except inlimited circumstance, such as to correct errors or improve operation, soas not to disrupt trader expectations. It will be appreciated thatdifferent products offered by a particular exchange may use differentmatching algorithms.

For example, a first-in/first-out (FIFO) matching algorithm, alsoreferred to as a “Price Time” algorithm, considers each identified ordersequentially in accordance with when the identified order was received.The quantity of the incoming order is matched to the quantity of theidentified order at the best price received earliest, then quantities ofthe next earliest best price orders, and so on until the quantity of theincoming order is exhausted. Some product specifications define the useof a pro-rata matching algorithm, wherein a quantity of an incomingorder is allocated to each of plurality of identified ordersproportionally. Some exchange computer systems provide a priority tocertain standing orders in particular markets. An example of such anorder is the first order that improves a price (i.e., improves themarket) for the product during a trading session. To be given priority,the trading platform may require that the quantity associated with theorder is at least a minimum quantity. Further, some exchange computersystems cap the quantity of an incoming order that is allocated to astanding order on the basis of a priority for certain markets. Inaddition, some exchange computer systems may give a preference to orderssubmitted by a trader who is designated as a market maker for theproduct. Other exchange computer systems may use other criteria todetermine whether orders submitted by a particular trader are given apreference. Typically, when the exchange computer system allocates aquantity of an incoming order to a plurality of identified orders at thesame price, the trading host allocates a quantity of the incoming orderto any orders that have been given priority. The exchange computersystem thereafter allocates any remaining quantity of the incoming orderto orders submitted by traders designated to have a preference, and thenallocates any still remaining quantity of the incoming order using theFIFO or pro-rata algorithms. Pro-rata algorithms used in some marketsmay require that an allocation provided to a particular order inaccordance with the pro-rata algorithm must meet at least a minimumallocation quantity. Any orders that do not meet or exceed the minimumallocation quantity are allocated to on a FIFO basis after the pro-rataallocation (if any quantity of the incoming order remains). Moreinformation regarding order allocation may be found in U.S. Pat. No.7,853,499, the entirety of which is incorporated by reference herein andrelied upon. Other examples of matching algorithms which may be definedfor allocation of orders of a particular financial product include:Price Explicit Time; Order Level Pro Rata; Order Level Priority ProRata; Preference Price Explicit Time; Preference Order Level Pro Rata;Preference Order Level Priority Pro Rata; Threshold Pro-Rata; PriorityThreshold Pro-Rata; Preference Threshold Pro-Rata; Priority PreferenceThreshold Pro-Rata; and Split Price-Time Pro-Rata, which are describedin U.S. patent application Ser. No. 13/534,499, filed on Jun. 27, 2012,entitled “Multiple Trade Matching Algorithms,” published as U.S. PatentApplication Publication No. 2014/0006243 A1, the entirety of which isincorporated by reference herein and relied upon.

With respect to incoming orders, some traders, such as automated and/oralgorithmic traders, attempt to respond to market events, such as tocapitalize upon a mispriced resting order or other market inefficiency,as quickly as possible. This may result in penalizing the trader whomakes an errant trade, or whose underlying trading motivations havechanged, and who cannot otherwise modify or cancel their order fasterthan other traders can submit trades there against. It may consideredthat an electronic trading system that rewards the trader who submitstheir order first creates an incentive to either invest substantialcapital in faster trading systems, participate in the marketsubstantially to capitalize on opportunities (aggressor side/lower risktrading) as opposed to creating new opportunities (market making/higherrisk trading), modify existing systems to streamline business logic atthe cost of trade quality, or reduce one's activities and exposure inthe market. The result may be a lesser quality market and/or reducedtransaction volume, and corresponding thereto, reduced fees to theexchange.

With respect to resting orders, allocation/matching suitable restingorders to match against an incoming order can be performed, as describedherein, in many different ways. Generally, it will be appreciated thatallocation/matching algorithms are only needed when the incoming orderquantity is less than the total quantity of the suitable resting ordersas, only in this situation, is it necessary to decide which restingorder(s) will not be fully satisfied, which trader(s) will not get theirorders filled. It can be seen from the above descriptions of thematching/allocation algorithms, that they fall generally into threecategories: time priority/first-in-first-out (“FIFO”), pro rata, or ahybrid of FIFO and pro rata.

FIFO generally rewards the first trader to place an order at aparticular price and maintains this reward indefinitely. So if a traderis the first to place an order at price X, no matter how long that orderrests and no matter how many orders may follow at the same price, assoon as a suitable incoming order is received, that first trader will bematched first. This “first mover” system may commit other traders topositions in the queue after the first move traders. Furthermore, whileit may be beneficial to give priority to a trader who is first to placean order at a given price because that trader is, in effect, taking arisk, the longer that the trader's order rests, the less beneficial itmay be. For instance, it could deter other traders from adding liquidityto the marketplace at that price because they know the first mover (andpotentially others) already occupies the front of the queue.

With a pro rata allocation, incoming orders are effectively split amongsuitable resting orders. This provides a sense of fairness in thateveryone may get some of their order filled. However, a trader who tooka risk by being first to place an order (a “market turning” order) at aprice may end up having to share an incoming order with a much latersubmitted order. Furthermore, as a pro rata allocation distributes theincoming order according to a proportion based on the resting orderquantities, traders may place orders for large quantities, which theyare willing to trade but may not necessarily want to trade, in order toincrease the proportion of an incoming order that they will receive.This results in an escalation of quantities on the order book andexposes a trader to a risk that someone may trade against one of theseorders and subject the trader to a larger trade than they intended. Inthe typical case, once an incoming order is allocated against theselarge resting orders, the traders subsequently cancel the remainingresting quantity which may frustrate other traders. Accordingly, as FIFOand pro rata both have benefits and problems, exchanges may try to usehybrid allocation/matching algorithms which attempt to balance thesebenefits and problems by combining FIFO and pro rata in some manner.However, hybrid systems define conditions or fixed rules to determinewhen FIFO should be used and when pro rata should be used. For example,a fixed percentage of an incoming order may be allocated using a FIFOmechanism with the remainder being allocated pro rata.

Spread Instruments

Traders trading on an exchange including, for example, exchange computersystem 100, often desire to trade multiple financial instruments incombination. Each component of the combination may be called a leg.Traders can submit orders for individual legs or in some cases cansubmit a single order for multiple financial instruments in anexchange-defined combination. Such orders may be called a strategyorder, a spread order, or a variety of other names.

A spread instrument may involve the simultaneous purchase of onesecurity and sale of a related security, called legs, as a unit. Thelegs of a spread instrument may be options or futures contracts, orcombinations of the two. Trades in spread instruments are executed toyield an overall net position whose value, called the spread, depends onthe difference between the prices of the legs. Spread instruments may betraded in an attempt to profit from the widening or narrowing of thespread, rather than from movement in the prices of the legs directly.Spread instruments are either “bought” or “sold” depending on whetherthe trade will profit from the widening or narrowing of the spread,respectively. An exchange often supports trading of common spreads as aunit rather than as individual legs, thus ensuring simultaneousexecution of the two legs, eliminating the execution risk of one legexecuting but the other failing.

One example of a spread instrument is a calendar spread instrument. Thelegs of a calendar spread instrument differ in delivery date of theunderlier. The leg with the earlier occurring delivery date is oftenreferred to as the lead month contract. A leg with a later occurringdelivery date is often referred to as a deferred month contract. Anotherexample of a spread instrument is a butterfly spread instrument, whichincludes three legs having different delivery dates. The delivery datesof the legs may be equidistant to each other. The counterparty ordersthat are matched against such a combination order may be individual,“outright” orders or may be part of other combination orders.

In other words, an exchange may receive, and hold or let rest on thebooks, outright orders for individual contracts as well as outrightorders for spreads associated with the individual contracts. An outrightorder (for either a contract or for a spread) may include an outrightbid or an outright offer, although some outright orders may bundle manybids or offers into one message (often called a mass quote).

A spread is an order for the price difference between two contracts.This results in the trader holding a long and a short position in two ormore related futures or options on futures contracts, with the objectiveof profiting from a change in the price relationship. A typical spreadproduct includes multiple legs, each of which may include one or moreunderlying financial instruments. A butterfly spread product, forexample, may include three legs. The first leg may consist of buying afirst contract. The second leg may consist of selling two of a secondcontract. The third leg may consist of buying a third contract. Theprice of a butterfly spread product may be calculated as:

Butterfly=Leg1−2×Leg2+Leg3  (equation 1)

In the above equation, Leg1 equals the price of the first contract, Leg2equals the price of the second contract and Leg3 equals the price of thethird contract. Thus, a butterfly spread could be assembled from twointer-delivery spreads in opposite directions with the center deliverymonth common to both spreads.

A calendar spread, also called an intra-commodity spread, for futures isan order for the simultaneous purchase and sale of the same futurescontract in different contract months (i.e., buying a September CME S&P500® futures contract and selling a December CME S&P 500 futurescontract).

A crush spread is an order, usually in the soybean futures market, forthe simultaneous purchase of soybean futures and the sale of soybeanmeal and soybean oil futures to establish a processing margin. A crackspread is an order for a specific spread trade involving simultaneouslybuying and selling contracts in crude oil and one or more derivativeproducts, typically gasoline and heating oil. Oil refineries may trade acrack spread to hedge the price risk of their operations, whilespeculators attempt to profit from a change in the oil/gasoline pricedifferential.

A straddle is an order for the purchase or sale of an equal number ofputs and calls, with the same strike price and expiration dates. A longstraddle is a straddle in which a long position is taken in both a putand a call option. A short straddle is a straddle in which a shortposition is taken in both a put and a call option. A strangle is anorder for the purchase of a put and a call, in which the options havethe same expiration and the put strike is lower than the call strike,called a long strangle. A strangle may also be the sale of a put and acall, in which the options have the same expiration and the put strikeis lower than the call strike, called a short strangle. A pack is anorder for the simultaneous purchase or sale of an equally weighted,consecutive series of four futures contracts, quoted on an average netchange basis from the previous day's settlement price. Packs provide areadily available, widely accepted method for executing multiple futurescontracts with a single transaction. A bundle is an order for thesimultaneous sale or purchase of one each of a series of consecutivefutures contracts. Bundles provide a readily available, widely acceptedmethod for executing multiple futures contracts with a singletransaction.

Implication

Thus, an exchange may match outright orders, such as individualcontracts or spread orders (which as discussed herein could includemultiple individual contracts). The exchange may also imply orders fromoutright orders. For example, exchange computer system 100 may derive,identify and/or advertise, publish, display or otherwise make availablefor trading orders based on outright orders.

As was described above, the financial instruments which are the subjectof the orders to trade, may include one or more component financialinstruments. While each financial instrument may have its own orderbook, i.e., market, in which it may be traded, in the case of afinancial instrument having more than one component financialinstrument, those component financial instruments may further have theirown order books in which they may be traded. Accordingly, when an orderfor a financial instrument is received, it may be matched against asuitable counter order in its own order book or, possibly, against acombination of suitable counter orders in the order books the componentfinancial instruments thereof, or which share a common componentfinancial instrument. For example, an order for a spread contractcomprising component financial instruments A and B may be matchedagainst another suitable order for that spread contract. However, it mayalso be matched against suitable separate counter orders for the A andfor the B component financial instruments found in the order bookstherefore. Similarly, if an order for the A contract is received andsuitable match cannot be found in the A order book, it may be possibleto match order for A against a combination of a suitable counter orderfor a spread contract comprising the A and B component financialinstruments and a suitable counter order for the B component financialinstrument. This is referred to as “implication” where a given order fora financial instrument may be matched via a combination of suitablecounter orders for financial instruments which share common, orotherwise interdependent, component financial instruments. Implicationincreases the liquidity of the market by providing additionalopportunities for orders to be traded. Increasing the number oftransactions may further increase the number of transaction feescollected by the electronic trading system.

The order for a particular financial instrument actually received from amarket participant, whether it comprises one or more component financialinstruments, is referred to as a “real” or “outright” order, or simplyas an outright. The one or more orders which must be synthesized andsubmitted into order books other than the order book for the outrightorder in order to create matches therein, are referred to as “implied”orders. Upon receipt of an incoming order, the identification orderivation of suitable implied orders which would allow at least apartial trade of the incoming outright order to be executed is referredto as “implication” or “implied matching”, the identified orders beingreferred to as an “implied match.” Depending on the number componentfinancial instruments involved, and whether those component financialinstruments further comprise component financial instruments of theirown, there may be numerous different implied matches identified whichwould allow the incoming order to be at least partially matched andmechanisms may be provided to arbitrate, e.g., automatically, amongthem, such as by picking the implied match comprising the least numberof component financial instruments or the least number of synthesizedorders.

Upon receipt of an incoming order, or thereafter, a combination of oneor more suitable/hypothetical counter orders which have not actuallybeen received but if they were received, would allow at least a partialtrade of the incoming order to be executed, may be, e.g., automatically,identified or derived and referred to as an “implied opportunity.” Aswith implied matches, there may be numerous implied opportunitiesidentified for a given incoming order. Implied opportunities areadvertised to the market participants, such as via suitable syntheticorders, e.g. counter to the desired order, being placed on therespective order books to rest (or give the appearance that there is anorder resting) and presented via the market data feed, electronicallycommunicated to the market participants, to appear available to trade inorder to solicit the desired orders from the market participants.Depending on the number component financial instruments involved, andwhether those component financial instruments further comprise componentfinancial instruments of their own, there may be numerous impliedopportunities, the submission of a counter order in response thereto,would allow the incoming order to be at least partially matched.

Implied opportunities, e.g., the advertised synthetic orders, mayfrequently have better prices than the corresponding real orders in thesame contract. This can occur when two or more traders incrementallyimprove their order prices in the hope of attracting a trade, sincecombining the small improvements from two or more real orders can resultin a big improvement in their combination. In general, advertisingimplied opportunities at better prices will encourage traders to enterthe opposing orders to trade with them. The more implied opportunitiesthat the match engine of an electronic trading system cancalculate/derive, the greater this encouragement will be and the morethe Exchange will benefit from increased transaction volume. However,identifying implied opportunities may be computationally intensive. In ahigh performance trading system where low transaction latency isimportant, it may be important to identify and advertise impliedopportunities quickly so as to improve or maintain market participantinterest and/or market liquidity.

For example, two different outright orders may be resting on the books,or be available to trade or match. The orders may be resting becausethere are no outright orders that match the resting orders. Thus, eachof the orders may wait or rest on the books until an appropriateoutright counteroffer comes into the exchange or is placed by a user ofthe exchange. The orders may be for two different contracts that onlydiffer in delivery dates. It should be appreciated that such orderscould be represented as a calendar spread order. Instead of waiting fortwo appropriate outright orders to be placed that would match the twoexisting or resting orders, the exchange computer system may identify ahypothetical spread order that, if entered into the system as a tradablespread order, would allow the exchange computer system to match the twooutright orders. The exchange may thus advertise or make available aspread order to users of the exchange system that, if matched with atradable spread order, would allow the exchange to also match the tworesting orders. Thus, the match engine is configured to detect that thetwo resting orders may be combined into an order in the spreadinstrument and accordingly creates an implied order.

In other words, the exchange's matching system may imply thecounteroffer order by using multiple orders to create the counterofferorder. Examples of spreads include implied IN, implied OUT, 2nd- ormultiple-generation, crack spreads, straddle, strangle, butterfly, andpack spreads. Implied IN spread orders are derived from existingoutright orders in individual legs. Implied OUT outright orders arederived from a combination of an existing spread order and an existingoutright order in one of the individual underlying legs. Implied orderscan fill in gaps in the market and allow spreads and outright futurestraders to trade in a product where there would otherwise have beenlittle or no available bids and asks.

For example, implied IN spreads may be created from existing outrightorders in individual contracts where an outright order in a spread canbe matched with other outright orders in the spread or with acombination of orders in the legs of the spread. An implied OUT spreadmay be created from the combination of an existing outright order in aspread and an existing outright order in one of the individualunderlying leg. An implied IN or implied OUT spread may be created whenan electronic match system simultaneously works synthetic spread ordersin spread markets and synthetic orders in the individual leg marketswithout the risk to the trader/broker of being double filled or filledon one leg and not on the other leg.

By linking the spread and outright markets, implied spread tradingincreases market liquidity. For example, a buy in one contract month andan offer in another contract month in the same futures contract cancreate an implied market in the corresponding calendar spread. Anexchange may match an order for a spread product with another order forthe spread product. Some existing exchanges attempt to match orders forspread products with multiple orders for legs of the spread products.With such systems, every spread product contract is broken down into acollection of legs and an attempt is made to match orders for the legs.

Implied orders, unlike real orders, are generated by electronic tradingsystems. In other words, implied orders are computer generated ordersderived from real orders. The system creates the “derived” or “implied”order and provides the implied order as a market that may be tradedagainst. If a trader trades against this implied order, then the realorders that combined to create the implied order and the resultingmarket are executed as matched trades. Implied orders generally increaseoverall market liquidity. The creation of implied orders increases thenumber of tradable items, which has the potential of attractingadditional traders. Exchanges benefit from increased transaction volume.Transaction volume may also increase as the number of matched tradeitems increases.

Examples of implied spread trading include those disclosed in U.S.Patent Publication No. 2005/0203826, entitled “Implied Spread TradingSystem,” the entire disclosure of which is incorporated by referenceherein and relied upon. Examples of implied markets include thosedisclosed in U.S. Pat. No. 7,039,610, entitled “Implied Market TradingSystem,” the entire disclosure of which is incorporated by referenceherein and relied upon.

In some cases, the outright market for the deferred month or otherconstituent contract may not be sufficiently active to provide marketdata (e.g., bid-offer data) and/or trade data. Spread instrumentsinvolving such contracts may nonetheless be made available by theexchange. The market data from the spread instruments may then be usedto determine a settlement price for the constituent contract. Thesettlement price may be determined, for example, through a boundaryconstraint-based technique based on the market data (e.g., bid-offerdata) for the spread instrument, as described in U.S. Patent PublicationNo. 2015/0073962 entitled “Boundary Constraint-Based Settlement inSpread Markets” (“the '962 Publication”), the entire disclosure of whichis incorporated by reference herein and relied upon. Settlement pricedetermination techniques may be implemented to cover calendar monthspread instruments having different deferred month contracts.

Order Book Object Data Structures

In one embodiment, the messages and/or values received for each objectmay be stored in queues according to value and/or priority techniquesimplemented by an exchange computing system 100. FIG. 3A illustrates anexample data structure 300, which may be stored in a memory or otherstorage device, such as the memory 204 or storage device 206 describedwith respect to FIG. 2 , for storing and retrieving messages related todifferent values for the same action for an object. For example, datastructure 300 may be a set of queues or linked lists for multiple valuesfor an action, e.g., bid, on an object. Data structure 300 may beimplemented as a database. It should be appreciated that the system maystore multiple values for the same action for an object, for example,because multiple users submitted messages to buy specified quantities ofan object at different values. Thus, in one embodiment, the exchangecomputing system may keep track of different orders or messages forbuying or selling quantities of objects at specified values.

Although the present patent application contemplates using queue datastructures for storing messages in a memory, the implementation mayinvolve additional pointers, i.e., memory address pointers, or linkingto other data structures. Incoming messages may be stored at anidentifiable memory address. The transaction processor can traversemessages in order by pointing to and retrieving different messages fromthe different memories. Thus, messages that may be depictedsequentially, e.g., in FIG. 3B below, may actually be stored in memoryin disparate locations. The software programs implementing thetransaction processing may retrieve and process messages in sequencefrom the various disparate (e.g., random) locations. Thus, in oneembodiment, each queue may store different values, which could representprices, where each value points to or is linked to the messages (whichmay themselves be stored in queues and sequenced according to prioritytechniques, such as prioritizing by value) that will match at thatvalue. For example, as shown in FIG. 3A, all of the values relevant toexecuting an action at different values for an object are stored in aqueue. Each value in turn points to, e.g., a linked list or queuelogically associated with the values. The linked list stores themessages that instruct the exchange computing system to buy specifiedquantities of the object at the corresponding value.

The sequence of the messages in the message queues connected to eachvalue may be determined by exchange implemented priority techniques. Forexample, in FIG. 3A, messages M1, M2, M3 and M4 are associated withperforming an action (e.g., buying or selling) a certain number of units(may be different for each message) at Value 1. M1 has priority over M2,which has priority over M3, which has priority over M4. Thus, if acounter order matches at Value 1, the system fills as much quantity aspossible associated with M1 first, then M2, then M3, and then M4.

In the illustrated examples, the values may be stored in sequentialorder, and the best or lead value for a given queue may be readilyretrievable by and/or accessible to the disclosed system. Thus, in oneembodiment, the value having the best priority may be illustrated asbeing in the topmost position in a queue, although the system may beconfigured to place the best priority message in some otherpredetermined position. In the example of FIG. 3A, Value 1 is shown asbeing the best value or lead value, or the top of the book value, for anexample Action.

FIG. 3B illustrates an example alternative data structure 350 forstoring and retrieving messages and related values. It should beappreciated that matches occur based on values, and so all the messagesrelated to a given value may be prioritized over all other messagesrelated to a different value. As shown in FIG. 3B, the messages may bestored in one queue and grouped by values according to the hierarchy ofthe values. The hierarchy of the values may depend on the action to beperformed.

For example, if a queue is a sell queue (e.g., the Action is Sell), thelowest value may be given the best priority and the highest value may begiven the lowest priority. Thus, as shown in FIG. 3B, if Value 1 islower than Value 2 which is lower than Value 3, Value 1 messages may beprioritized over Value 2, which in turn may be prioritized over Value 3.

Within Value 1, M1 is prioritized over M2, which in turn is prioritizedover M3, which in turn is prioritized over M4. Within Value 2, M5 isprioritized over M6, which in turn is prioritized over M7, which in turnis prioritized over M8. Within Value 3, M9 is prioritized over M10,which in turn is prioritized over M11, which in turn is prioritized overM12.

Alternatively, the messages may be stored in a tree-node data structurethat defines the priorities of the messages. In one embodiment, themessages may make up the nodes.

In one embodiment, the system may traverse through a number of differentvalues and associated messages when processing an incoming message.Traversing values may involve the processor loading each value, checkingthat value and deciding whether to load another value, i.e., byaccessing the address pointed at by the address pointer value. Inparticular, referring to FIG. 3B, if the queue is for selling an objectfor the listed Values 1, 2 and 3 (where Value 1 is lower than Value 2which is lower than Value 3), and if the system receives an incomingaggressing order to buy quantity X at a Value 4 that is greater thanValues 1, 2, and 3, the system will fill as much of quantity X aspossible by first traversing through the messages under Value 1 (insequence M1, M2, M3, M4). If any of the quantity of X remains, thesystem traverses down the prioritized queue until all of the incomingorder is filled (e.g., all of X is matched) or until all of thequantities of M1 through M12 are filled. Any remaining, unmatchedquantity remains on the books, e.g., as a resting order at Value 4,which was the entered value or the message's value.

The system may traverse the queues and check the values in a queue, andupon finding the appropriate value, may locate the messages involved inmaking that value available to the system. When an outright messagevalue is stored in a queue, and when that outright message is involvedin a trade or match, the system may check the queue for the value, andthen may check the data structure storing messages associated with thatvalue.

In one embodiment, an exchange computing system may convert allfinancial instruments to objects. In one embodiment, an object mayrepresent the order book for a financial instrument. Moreover, in oneembodiment, an object may be defined by two queues, one queue for eachaction that can be performed by a user on the object. For example, anorder book converted to an object may be represented by an Ask queue anda Bid queue. Resting messages or orders associated with the respectivefinancial instrument may be stored in the appropriate queue and recalledtherefrom.

In one embodiment, the messages associated with objects may be stored inspecific ways depending on the characteristics of the various messagesand the states of the various objects in memory. For example, a systemmay hold certain resting messages in queue until the message is to beprocessed, e.g., the message is involved in a match. The order, sequenceor priority given to messages may depend on the characteristics of themessage. For example, in certain environments, messages may indicate anaction that a computer in the system should perform. Actions may becomplementary actions, or require more than one message to complete. Forexample, a system may be tasked with matching messages or actionscontained within messages. The messages that are not matched may bequeued by the system in a data queue or other structure, e.g., a datatree having nodes representing messages or orders.

The queues are structured so that the messages are stored in sequenceaccording to priority. Although the embodiments are disclosed as beingimplemented in queues, it should be understood that different datastructures such as, for example, linked lists or trees may also be used.

The system may include separate data structures, e.g., queues, fordifferent actions associated with different objects within the system.For example, in one embodiment, the system may include a queue for eachpossible action that can be performed on an object. The action may beassociated with a value. The system prioritizes the actions based inpart on the associated value.

For example, as shown in FIG. 3C, the order book module of a computingsystem may include several paired queues, such as queues Bid and Ask foran object 302 (e.g., Object A). The system may include two queues, orone pair of queues, for each object that is matched or processed by thesystem. In one embodiment, the system stores messages in the queues thathave not yet been matched or processed. FIG. 3C may be an implementationof the data structures disclosed in FIGS. 3A and/or 3B. Each queue mayhave a top of book, or lead, position, such as positions 304 and 306,which stores data that is retrievable.

The queues may define the priority or sequence in which messages areprocessed upon a match event. For example, two messages stored in aqueue may represent performing the same action at the same value. When athird message is received by the system that represents a matchingaction at the same value, the system may need to select one of the twowaiting, or resting, messages as the message to use for a match. Thus,when multiple messages can be matched at the same value, the exchangemay have a choice or some flexibility regarding the message that ismatched. The queues may define the priority in which orders that areotherwise equivalent (e.g., same action for the same object at the samevalue) are processed.

The system may include a pair of queues for each object, e.g., a bid andask queue for each object. Each queue may be, for example, implementedutilizing the data structure of FIG. 3B. The exchange may be able tospecify the conditions upon which a message for an object should beplaced in a queue. For example, the system may include one queue foreach possible action that can be performed on an object. The system maybe configured to process messages that match with each other. In oneembodiment, a message that indicates performing an action at a value maymatch with a message indicating performing a corresponding action at thesame value. Or, the system may determine the existence of a match whenmessages for the same value exist in both queues of the same object. Themessages may be received from the same or different users or traders.

The queues illustrated in FIG. 3C hold or store messages received by acomputing exchange, e.g., messages submitted by a user to the computingexchange, and waiting for a proper match. It should be appreciated thatthe queues may also hold or store implieds, e.g., implied messagesgenerated by the exchange system, such as messages implied in or impliedout as described herein. The system thus adds messages to the queues asthey are received, e.g., messages submitted by users, or generated,e.g., implied messages generated by the exchanges. The sequence orprioritization of messages in the queues is based on information aboutthe messages and the overall state of the various objects in the system.

When the data transaction processing system is implemented as anexchange computing system, as discussed above, different clientcomputers submit electronic data transaction request messages to theexchange computing system. Electronic data transaction request messagesinclude requests to perform a transaction on a data object, e.g., at avalue for a quantity. The exchange computing system includes atransaction processor, e.g., a hardware matching processor or matchengine, that matches, or attempts to match, pairs of messages with eachother. For example, messages may match if they contain counterinstructions (e.g., one message includes instructions to buy, the othermessage includes instructions to sell) for the same product at the samevalue. In some cases, depending on the nature of the message, the valueat which a match occurs may be the submitted value or a better value. Abetter value may mean higher or lower value depending on the specifictransaction requested. For example, a buy order may match at thesubmitted buy value or a lower (e.g., better) value. A sell order maymatch at the submitted sell value or a higher (e.g., better) value.

Transaction Processor Data Structures

FIG. 4 illustrates an example embodiment of a data structure used toimplement match engine module 106. Match engine module 106 may include aconversion component 402, pre-match queue 404, match component 406,post-match queue 408 and publish component 410.

Although the embodiments are disclosed as being implemented in queues,it should be understood that different data structures, such as forexample linked lists or trees, may also be used. Although theapplication contemplates using queue data structures for storingmessages in a memory, the implementation may involve additionalpointers, i.e., memory address pointers, or linking to other datastructures. Thus, in one embodiment, each incoming message may be storedat an identifiable memory address. The transaction processing componentscan traverse messages in order by pointing to and retrieving differentmessages from the different memories. Thus, messages that may beprocessed sequentially in queues may actually be stored in memory indisparate locations. The software programs implementing the transactionprocessing may retrieve and process messages in sequence from thevarious disparate (e.g., random) locations.

The queues described herein may, in one embodiment, be structured sothat the messages are stored in sequence according to time of receipt,e.g., they may be first in first out (FIFO) queues.

The match engine module 106 may be an example of a transactionprocessing system. The pre-match queue 404 may be an example of apre-transaction queue. The match component 406 may be an example of atransaction component. The post-match queue 408 may be an example of apost-transaction queue. The publish component 410 may be an example of adistribution component. The transaction component may process messagesand generate transaction component results.

In one embodiment, the publish component may be a distribution componentthat can distribute data to one or more market participant computers. Inone embodiment, match engine module 106 operates according to a firstin, first out (FIFO) ordering. The conversion component 402 converts orextracts a message received from a trader via the Market Segment Gatewayor MSG into a message format that can be input into the pre-match queue404.

Messages from the pre-match queue may enter the match component 406sequentially and may be processed sequentially. In one regard, thepre-transaction queue, e.g., the pre-match queue, may be considered tobe a buffer or waiting spot for messages before they can enter and beprocessed by the transaction component, e.g., the match component. Thematch component matches orders, and the time a message spends beingprocessed by the match component can vary, depending on the contents ofthe message and resting orders on the book. Thus, newly receivedmessages wait in the pre-transaction queue until the match component isready to process those messages. Moreover, messages are received andprocessed sequentially or in a first-in, first-out FIFO methodology. Thefirst message that enters the pre-match or pre-transaction queue will bethe first message to exit the pre-match queue and enter the matchcomponent. In one embodiment, there is no out-of-order messageprocessing for messages received by the transaction processing system.The pre-match and post-match queues are, in one embodiment, fixed insize, and any messages received when the queues are full may need towait outside the transaction processing system or be re-sent to thetransaction processing system.

The match component 406 processes an order or message, at which pointthe transaction processing system may consider the order or message ashaving been processed. The match component 406 may generate one messageor more than one message, depending on whether an incoming order wassuccessfully matched by the match component. An order message thatmatches against a resting order in the order book may generate dozens orhundreds of messages. For example, a large incoming order may matchagainst several smaller resting orders at the same price level. Forexample, if many orders match due to a new order message, the matchengine needs to send out multiple messages informing traders whichresting orders have matched. Or, an order message may not match anyresting order and only generates an acknowledgement message. Thus, thematch component 406 in one embodiment will generate at least onemessage, but may generate more messages, depending upon the activitiesoccurring in the match component. For example, the more orders that arematched due to a given message being processed by the match component,the more time may be needed to process that message. Other messagesbehind that given message will have to wait in the pre-match queue.

Messages resulting from matches in match component 406 enter thepost-match queue 408. The post-match queue may be similar infunctionality and structure to the pre-match queue discussed above,e.g., the post-match queue is a FIFO queue of fixed size. As illustratedin FIG. 4 , a difference between the pre- and post-match queues may bethe location and contents of the structures, namely, the pre-match queuestores messages that are waiting to be processed, whereas the post-matchqueue stores match component results due to matching by the matchcomponent. The match component receives messages from the pre-matchqueue, and sends match component results to the post-match queue. In oneembodiment, the time that results messages, generated due to thetransaction processing of a given message, spend in the post-match queueis not included in the latency calculation for the given message.

Messages from the post-match queue 408 enter the publish component 410sequentially and are published via the MSG sequentially. Thus, themessages in the post-match queue 408 are an effect or result of themessages that were previously in the pre-match queue 404. In otherwords, messages that are in the pre-match queue 404 at any given timewill have an impact on or affect the contents of the post-match queue408, depending on the events that occur in the match component 406 oncethe messages in the pre-match queue 404 enter the match component 406.

As noted above, the match engine module 106 in one embodiment operatesin a first in first out (FIFO) scheme. In other words, the first messagethat enters the match engine module 106 is the first message that isprocessed by the match engine module 106. Thus, the match engine module106 in one embodiment processes messages in the order the messages arereceived. In FIG. 4 , as shown by the data flow arrow, data is processedsequentially by the illustrated structures from left to right, beginningat the conversion component 402, to the pre-match queue, to the matchcomponent 406, to the post-match queue 408, and to the publish component410. The overall transaction processing system operates in a FIFO schemesuch that data flows from element 402 to 404 to 406 to 408 to 410, inthat order. If any one of the queues or components of the transactionprocessing system experiences a delay, that creates a backlog for thestructures preceding the delayed structure. For example, if the match ortransaction component is undergoing a high processing volume, and if thepre-match or pre-transaction queue is full of messages waiting to enterthe match or transaction component, the conversion component may not beable to add any more messages to the pre-match or pre-transaction queue.

Messages wait in the pre-match queue. The time a message waits in thepre-match queue depends upon how many messages are ahead of that message(i.e., earlier messages), and how much time each of the earlier messagesspends being serviced or processed by the match component. Messages alsowait in the post-match queue. The time a message waits in the post-matchqueue depends upon how many messages are ahead of that message (i.e.,earlier messages), and how much time each of the earlier messages spendsbeing serviced or processed by the publish component. These wait timesmay be viewed as a latency that can affect a market participant'strading strategy.

After a message is published (after being processed by the componentsand/or queues of the match engine module), e.g., via a market data feed,the message becomes public information and is publicly viewable andaccessible. Traders consuming such published messages may act upon thosemessages, e.g., submit additional new input messages to the exchangecomputing system responsive to the published messages.

The match component attempts to match aggressing or incoming ordersagainst resting orders. If an aggressing order does not match anyresting orders, then the aggressing order may become a resting order, oran order resting on the books. For example, if a message includes a neworder that is specified to have a one-year time in force, and the neworder does not match any existing resting order, the new order willessentially become a resting order to be matched (or attempted to bematched) with some future aggressing order. The new order will thenremain on the books for one year. On the other hand, an order specifiedas a fill or kill (e.g., if the order cannot be filled or matched withan order currently resting on the books, the order should be canceled)will never become a resting order, because it will either be filled ormatched with a currently resting order, or it will be canceled. Theamount of time needed to process or service a message once that messagehas entered the match component may be referred to as a service time.The service time for a message may depend on the state of the orderbooks when the message enters the match component, as well as thecontents, e.g., orders, that are in the message.

In one embodiment, orders in a message are considered to be “locked in”,or processed, or committed, upon reaching and entering the matchcomponent. If the terms of the aggressing order match a resting orderwhen the aggressing order enters the match component, then theaggressing order will be in one embodiment guaranteed to match.

As noted above, the latency experienced by a message, or the amount oftime a message spends waiting to enter the match component, depends uponhow many messages are ahead of that message (i.e., earlier messages),and how much time each of the earlier messages spends being serviced orprocessed by the match component. The amount of time a match componentspends processing, matching or attempting to match a message dependsupon the type of message, or the characteristics of the message. Thetime spent inside the processor may be considered to be a service time,e.g., the amount of time a message spends being processed or serviced bythe processor.

The number of matches or fills that may be generated in response to anew order message for a financial instrument will depend on the state ofthe data object representing the electronic marketplace for thefinancial instrument. The state of the match engine can change based onthe contents of incoming messages.

It should be appreciated that the match engine's overall latency is inpart a result of the match engine processing the messages it receives.The match component's service time may be a function of the message type(e.g., new, modify, cancel), message arrival rate (e.g., how many ordersor messages is the match engine module receiving, e.g., messages persecond), message arrival time (e.g., the time a message hits the inboundMSG or market segment gateway), number of fills generated (e.g., howmany fills were generated due to a given message, or how many ordersmatched due to an aggressing or received order), or number of Mass Quoteentries (e.g., how many of the entries request a mass quote).

In one embodiment, the time a message spends:

Being converted in the conversion component 402 may be referred to as aconversion time;

Waiting in the pre-match queue 404 may be referred to as a wait untilmatch time;

Being processed or serviced in the match component 406 may be referredto as a matching time;

Waiting in the post-match queue 408 may be referred to as a wait untilpublish time; and

Being processed or published via the publish component 410 may bereferred to as a publishing time.

It should be appreciated that the latency may be calculated, in oneembodiment, as the sum of the conversion time and wait until match time.Or, the system may calculate latency as the sum of the conversion time,wait until match time, matching time, wait until publish time, andpublishing time. In systems where some or all of those times arenegligible, or consistent, a measured latency may only include the sumof some of those times. Or, a system may be designed to only calculateone of the times that is the most variable, or that dominates (e.g.,percentage wise) the overall latency. For example, some marketparticipants may only care about how long a newly sent message that isadded to the end of the pre-match queue will spend waiting in thepre-match queue. Other market participants may care about how long thatmarket participant will have to wait to receive an acknowledgement fromthe match engine that a message has entered the match component. Yetother market participants may care about how much time will pass fromwhen a message is sent to the match engine's conversion component towhen match component results exit or egress from the publish component.

Monitoring System Data Store

A data store contains information, e.g., data entries, recorded thereonby a monitoring system that monitors processing of messages in a datatransaction processing system. The monitoring system may be implementedto measure the performance of a computer system, such as a match engineof an electronic trading system, whereby multiple threads, processes orother programs, referred to as “loggers” or “logging threads”, monitorvarious portions of the operation of the computer system and record,e.g., periodically, their observations, and/or other data indicativethereof, in a log data file or other data structure. The monitoringsystem may be implemented by multiple operating processes, threads,tasks or other computer program code construct, logically distributed orotherwise coupled throughout the exchange computer system 100 to monitordifferent parts, e.g., modules, thereof and record data regarding theoperation thereof in a log file or other data store. The data store mayinclude one or more data files, records or other structures or resourcesfor storing data.

The messages processed by the monitored system, e.g., an application,may be referred to as transactions. It should be appreciated that themore granular the monitoring, i.e., the more monitoring threads/processthat can be implemented to monitor more parts of the system and/or therate or frequency at which those parts may be monitored and dataindicative thereof recorded, the more useful the overall monitoringfunction may be. Each thread/process may be monitoring a differentportion of the system and the computer system may be operating at highspeed, thereby generating a significant amount of monitored data in ashort amount of time, all of which may be written by multiple threads toa same file or data store.

In one embodiment, the monitored system may comprise a match engine ofan electronic trading system and the monitoring threads/processes maycontinuously monitor and record time stamp and data about the locationin the code, which may be checkpoint data, indicative of or based on thetiming of particular operations/functions/milestones/states of thesystem for the purpose of system performance evaluation, problem/defectresolutions, historical data analysis, operational scenarioreconstruction, regulatory or administrative audit, or other purpose.The disclosed embodiments may be utilized in conjunction with any systemor multi-threaded implementation that enables multiple processes/threadsto append data to a shared resource, e.g. file or other data store, suchas U.S. patent application Ser. No. 15/663,360 entitled “ConcurrentWrite Operations For Use With Multi-Threaded File Logging” and filed onJul. 28, 2017 (“the '360 application”), assigned to the assignee of thepresent application, the entire disclosure of which is incorporated byreference herein and relied upon.

The application may be a software application. An application may beimplemented as software code, where different portions of the code maybe identifiable as, for example, different line numbers.

In one embodiment, a checkpoint may be one or more lines in the codethat instructs or causes the application to store information in a datastore, such as data store 510, such as code which implements or calls adata storage operation. For example, referring back to FIG. 4 , anapplication or module, such as the match engine module 106, may includecheckpoints 412 and 414.

A checkpoint inserted at a point or line in the code may include or beidentifiable as a checkpoint name. For example, checkpoint 412 may benamed a start checkpoint, or be named “pre-match component checkpoint”.When the checkpoint data is logged to the file, the logged data may listthe time a message reached the checkpoint called “pre-match componentcheckpoint”.

A message progresses through the code associated with the application.When a message traverses a checkpoint, such as checkpoints 412 or 414,the monitoring system logs information about the occurrence of thetraversal event by recording data indicative of the message, thecheckpoint name, and a timestamp associated with the occurrence of theevent.

A monitoring system may store, i.e., append, data, such as log data, ina selected one of at least one data store 510 discussed below, e.g., adata file or other data storage construct, and may be coupled with aprocessor, such as processor 202. The selected one of the at least onedata store 510 may be stored in a memory 204 or elsewhere. The processor202, memory 204 and/or data store 510 may be implemented by a processor202 and memory 204 as described herein with respect to FIG. 2 .

In one embodiment, the checkpoints may be implemented via a separate,supervisory, thread/process that watches the progress of a messagethrough the code and, upon the message traversing a certain point in thecode, may log information about the state of the system at that time.

Database Performance Based Indexing

After the data is collected/logged in the data store, the data can bequeried, such as by a performance analysis tool. However, in a highvolume data transaction processing system that records transactiondetails for many transactions, the size of the data store candrastically slow down query response time. A typical reason for poorquery response time is the large size of a data store being queried.

For example, an exchange computing system application, e.g., a matchengine module, may process hundreds of millions of messages a day. Theapplication may include many checkpoints, where, as discussed above, adata entry may be recorded each time a message traverses a checkpoint.Such a large volume of data, while very useful for performance analysis,can severely slow down query response times.

The disclosed performance measurement indexing system determines, duringa post-processing phase, which portions of the application, e.g., basedon the data entries recorded in the data store, are bottlenecked.

FIG. 5 illustrates an example performance measurement indexing system500, which in one embodiment is implemented as part of the exchangecomputer system 100 described herein. In particular, FIG. 5 shows asystem 500 for indexing a data store including data indicative ofmessage processing by an application 512 in a data transactionprocessing system. Application 512 may be an application such as matchengine module 106.

The indexing system 500 includes a latency detection system 508 whichmay be implemented as a separate hardware component or as logic storedin the memory 204 and executable by the processor 202 to cause theprocessor 202 to determine whether one or more portions of theapplication is experiencing a delay. Whether or not a portion of theapplication is undergoing or experiencing a delay may be defined in avariety of ways. It should be appreciated that the latency detectionsystem 508 retrieves information about message processing based on how adelay is defined within the system.

Although the application contemplates measuring latency betweencheckpoints, the disclosed embodiments may rely on or use jittermeasurements (i.e., latency variation), bandwidth consumed versusutilized, or message throughput (raw number of messages being processedper unit of time), instead of latency measurement, to modify a datastore containing performance measurement data entries as discussedherein.

An application may include different stages, which may be differentfunctions or processes executed by the code. A message may skip a stage,or portions, of the code, depending on the results of a previous/earlierstage. Stages may be identified by a beginning and an end checkpoint ofthat stage. An application developer may place checkpoints in the codethat define, or form boundaries, around stages. For example, referringto FIG. 6 , application 600 in FIG. 6 includes application stage 1(defined by checkpoints 602 and 608), stage 2 (defined by checkpoints610 and 616), and stage 3 (defined by checkpoints 618 and 624).

For example, a portion/stage of the application may be defined asundergoing or experiencing a delay if the amount of time that elapsesfor a message to traverse checkpoints defining that portion exceeds apredefined, threshold amount of time. Referring back to FIG. 5 , thelatency detection system 508 detects the latency or time elapse betweenprocessing of messages between any two of the checkpoints in theapplication 512. For example, in an application that includes fourcheckpoints A, B, C and D, the latency detection system 508 maydetermine the amount of time elapsed between a message traversingcheckpoint A and the same message traversing checkpoint B. The latencydetection system 508 may determine the amount of time elapsed between amessage traversing any two of the checkpoints in the application. In oneembodiment, the latency detection system 508 may additionally comparethe latency (or elapsed time) between a message traversing two of thecheckpoints with an expected latency threshold for the same twocheckpoints. For example, the expected latency threshold for checkpointpair AB may be 5 microseconds. If the actual latency experienced by amessage, e.g., the amount of time elapsed, between the messagetraversing checkpoints A and B exceeds 5 microseconds, the latencydetection system 508 may conclude or detect that checkpoint pair AB isdelayed.

In one embodiment, a portion/stage of the application may be defined asundergoing or experiencing a delay if the amount of time that elapsesfor a message to traverse checkpoints defining that portion is higherthan the amount of time that elapses for a message to traversecheckpoints defining other portions, e.g., application stages. Thelatency detection system 508 may determine whether the time elapsedbetween checkpoints A and B is a disproportionally high percentage ofthe overall time needed for a message to be processed by theapplication. Thus, the latency detection system 508 may use a relativecomparison of the amount of time that elapses as a message is processedthrough the different stages. The stage, i.e., a portion of theapplication defined by checkpoints, that requires the most amount oftime to process a message is determined to be experiencing a delay, oris determined to be a bottleneck in the application.

In one embodiment, a portion/stage of the application may be defined asundergoing or experiencing a delay if the amount of time that elapsesfor a message to be serviced/processed by that stage of the applicationis more than an amount of time between new messages arriving to thatstage. In other words, the arrival rate of messages to a stage mayexceed the per message processing time for that stage. For example, astage may receive a new message to process each microsecond, but mayrequire 2 microseconds to process a message. That stage may then bedefined to be undergoing a delay, because the service time of that stageis greater than the arrival rate of messages to that stage.

In one embodiment, a portion/stage of the application may be defined asundergoing or experiencing a delay if the number of messages waiting tobe serviced/processed by that stage of the application, i.e., the queuedepth, exceeds a predefined, threshold number of messages. For example,referring back to FIG. 4 , if the match component 406 is a stage of thematch engine module 106 application, the pre-match queue 404 holdsmessages that are waiting to be serviced by the match component 406. Ifthe pre-match queue contains more messages than a threshold amount ofmessages, the match component 406 may be considered to be experiencing adelay.

It should be appreciated that the conditions that define whether aportion of the application is undergoing a delay may be configurable bya user. In one embodiment, the monitoring system increases the number ofcheckpoints within a portion determined to be experiencing a delay, sothat more data is collected from such portions and specific problemportions of the application can be further analyzed and potentiallymitigated, e.g., by providing more resources to bottlenecked components,by testing the portion of the code for race conditions/memory leaks,etc.

The latency detection system 508 may be a component of the indexingsystem. In one embodiment, the latency detection system 508 may be aseparate component outside of the indexing system.

It should also be appreciated that an application 512 may be processingmany different messages at the same time. Thus, different messages maybe at different stages within the code at the same time. For example,concurrent logging of multi-threaded applications is discussed infurther detail in the '360 application. Thus, the data writer 504 maycontinuously be writing data related to different messages to the datastore 510. The message processing latency determined by the latencydetection system 508 is based on the time elapsed between one message,i.e., the same message, traversing two different checkpoints.

The latency detection system 508 may, in one embodiment, read data fromthe data store 510 to determine whether checkpoint pairs areexperiencing a higher than expected latency.

In one embodiment, the latency detection system 508 may only determinethe message processing latencies between checkpoints in checkpoint pairsdefining stages. For example, the latency detection system 508 maydetermine the message processing latencies between checkpoints 602 and608 (which define stage 1 of application 600), checkpoints 610 and 616(which define stage 2 of application 600), and checkpoints 618 and 624(which define stage 3 of application 600). In one embodiment, thecheckpoint pairs between which the latency detection system 508 maydetermine message processing latencies is configurable by a user.

The message processing latency for a given stage may be determined basedon the time required/elapsed for a message, or the average timerequired/elapsed for a plurality of messages, to travel from onecheckpoint defining a beginning of the stage to the other checkpointdefining an end of the stage. The latency detection system 508 maydetermine that stage 3 is experiencing a delay that exceeds apredetermined expected latency threshold. The predetermined expectedlatency thresholds for the different stages may be based on historicalperformance of messages that were previously processed by the stages.

Or, the latency detection system 508 may detect that stage 3 is abottleneck in the code because of the time needed/elapsed for one ormore messages to be processed through the stage. In softwareengineering, application performance improvement can be achieved byreducing overall message processing latency. If additional datapoints/data entries are collected from the stage that requires the mosttime to process, that stage can be analyzed, better understood, andperformance thereof can thereby be improved, which in turn can reduceoverall message processing latency of an application.

The indexing system 500 includes a delay marker 502 which may beimplemented as a separate hardware component or as logic stored in thememory 204 and executable by the processor 202 to cause the processor202 to mark, e.g., with a flag, portions of the data entries defined bythe latency detection system 508 as being unexpectedly delayed, orcongested. The thresholds that are used to determine whether the messageprocessing latency between a checkpoint pair constitutes an unexpecteddelay, such that the data entries associated with the checkpoint pairare included in subsequent performance analysis tool queries. Forexample, a bottleneck may be detected based on the interval of time, orthe time elapsed, between a message traversing two checkpoints.

The indexing system 500 includes a query receiver 504 which may beimplemented as a separate hardware component or as logic stored in thememory 204 and executable by the processor 202 to cause the processor202 to receive a query to search the data store 510. A user may querythe data store for performance analysis. For example, a user mayimplement a performance analysis tool to query the data store forpercentiles associated with message processing. Or, a user may query thedata store for performance of an application stage over time. Aperformance analysis tool may include an option, such as a checkbox,that when checked, limits the field of a query to areas of theapplication that were experiencing a delay, as determined/defined by thelatency detection system. Such a query that limits the field of a queryto areas of the application that were experiencing a delay may bereferred to as a bottleneck limited query.

The indexing system 500 includes a data store searcher 506 which may beimplemented as a separate hardware component or as logic stored in thememory 204 and executable by the processor 202 to cause the processor202 to identify, and only search, portions of the data entries marked asbeing delayed by the delay marker 502.

The indexing system 500 includes a data store searcher 506 which may beimplemented as a separate hardware component or as logic stored in thememory 204 and executable by the processor 202 to cause the processor202 to identify, and only search, portions of the data entries marked asbeing delayed by the delay marker 502.

The indexing system 500 includes a query responder 512 which may beimplemented as a separate hardware component or as logic stored in thememory 204 and executable by the processor 202 to cause the processor202 to generate responses to queries received by the query receiver 504.For example, the indexing system may supply a response to a clientcomputer that submitted, e.g., via a performance analysis tool, a queryto search the data store 510. The response may be performance data aboutmessage processing by an application. The performance analysis tool mayvisually summarize performance data as timeseries and percentiledistributions.

For example, referring back to the example of FIG. 6 discussed above,based on the determination that stage 3 is a bottleneck in theapplication, or is experiencing a higher than expected delay, the delaymarker 502 may flag or mark checkpoint pair 618, 624 as delayed, or maymodify the data entries associated with checkpoint pair 618, 624 byadding data indicative of a delay. When a bottleneck limited query isrun on the data store 510, only checkpoints that are marked as delayedappear in the search results. In one embodiment, all checkpoints thatfall within a delayed checkpoint pair appear in the results in responseto a bottleneck limited query. For example, in one embodiment, a usermay use a performance analysis tool to query for performancemeasurements that occur within a specific time window. If theperformance analysis tool limits the query to bottlenecked areas only,only results that occurred between checkpoint pair 618 and 624 arereturned to the user. Measurements within the specified time window butrelated to other stages, e.g., stages 1 and 2, are not searched orreturned. Because data entries/measurements from checkpoints associatedwith stages 1 and 2 are not searched, the performance measurementindexing system enables a much faster query response time.

In one embodiment, all of the checkpoints that appear between delayedcheckpoints are included in the results. For example, if checkpoint pair618, 624 is marked as delayed, but checkpoint pair 620, 622 is notmarked as delayed, a bottleneck limited query may search for and returnrelated to all of the checkpoints within stage 3, namely, 618, 620, 622and 624.

In one embodiment, the performance measurement indexing system may allowa user to set more than one delay condition for an application. Forexample, a user/software developer may set high and low urgency levelsfor delay conditions for stages in an application. For example,referring back to FIG. 6 , a developer may set two delay conditions forthe stages in application 600: 7 microseconds (high urgency) and 4microseconds (low urgency). The delay marker 502 marks checkpoint pairsas high or low urgency based on the amount of time recorded for messageprocessing latency between each stage defining checkpoint pair inapplication 600. Delay marker 502 accordingly does not mark stage 1 atall, marks stage 2 as low urgency and marks stage 3 as high urgency.When a user queries data store's data entries associated withapplication 600 using a performance analysis tool, the user may be ableto query for high and low urgency measurements. High urgency limitedqueries only search for data with stage 3, whereas low urgency limitedqueries only search for data within stages 2 and 3.

In one embodiment, the monitoring system may reuse identifiers to reducethe amount of data that is collected, as described in U.S. patentapplication Ser. No. 15/691,052 entitled “Compressed Message Tracing andParsing” and filed on Aug. 30, 2017 (“the '052 application”), assignedto the assignee of the present application, the entire disclosure ofwhich is incorporated by reference herein and relied upon. If themonitoring system reuses identifiers as described in, the indexingsystem parses, e.g., during post-processing, or after a message has beenprocessed by the application, the data structure to determine tracerentries associated with the same message for performance analysis of theprogress of the message through the code by identifying special markersor checkpoints traversed by the message. For example, the indexingsystem may parse the data store 510 as described in the '052application.

It should be appreciated that it may not be feasible or desirable toanalyze the data entries stored in the log file until a later time,e.g., when the system is no longer accepting messages. For example, inan exchange computing system, the logged data may not be reviewed forperformance analysis until the end of the trading day, after theexchange computing system stops accepting new messages. Thus, while thedata store is recording information about the plurality of messages, itmay not be efficient for an analysis tool to read the recorded data, forperformance analysis of messages within the code. Accordingly, theindexing system may index the data store when the data store is notactively storing new information. Alternatively, the indexing system mayindex the data store as the application 600 is processing new messages.In one embodiment, the indexing system only needs to be able tocalculate message processing latencies between checkpoints in checkpointpairs to determine if a checkpoint should be flagged, as discussedherein.

FIG. 7A illustrates an example data store 700 including data entries,e.g., data entries 702 through 724, tracing the progress of messagesthrough an application, such as application 600. The informationrecorded for each data entry may include an identifier 730 thatidentifies the message being processed, checkpoint information 732(e.g., which checkpoint in the application the message has traversed)and timestamp information 734 based on when the checkpoint is traversedby a particular message.

A monitoring system may begin recording entries in data store 700 asmessages M1 and M2 are received and processed by application 600. Again,a message traverses one or more of the checkpoints in an application asthe message is processed by the application. Data entries are added, orappended, to the data store 700 as checkpoint information for differentmessages is collected and recorded by a monitoring system. For example,data entry 702 indicates that message M1 reached or traversed thecheckpoint 602 at time 1 microsecond. Data entry 704 indicates thatmessage M1 reached checkpoint 608 at time 1.5 microseconds, or 0.5microseconds after reaching checkpoint 602. As indicated by data entry706, message M2 traverses checkpoint 602 at time 3 microseconds, and soon as indicated in FIG. 7A. Thus, the data entries shown in data store700 indicate the times that messages reached or traversed variouscheckpoints in the application. Each new data entry is appended to theend of the data store 700 in sequence as the data is collected. Theapplication may accordingly process messages M1 and M2 concurrently,i.e., the messages are in different stages within the application, andthe application has not completed processing any of those messages. Themonitoring system concurrently logs data about in the same data store,interweaving data entries about different messages as the differentmessages reach the various checkpoints within the code.

After the application 600 completes processing messages M1 and M2, thelatency detection system 508 within indexing system 500 may determinethe message processing latencies associated with each stage. FIG. 7Billustrates example message processing latencies by stage for messagesprocessed by an application. As shown in FIG. 7B, the latency detectionsystem 508 may determine the actual latencies experienced by messages M1and M2 as they were processed by the application 600. For example, themessage processing latency for M1 through Stage 1 (defined by checkpointpair 602, 608) may be determined as the timestamp for M1 traversingcheckpoint 608 (1.5 microseconds) minus the timestamp for M1 traversingcheckpoint 602 (1 microsecond), or 0.5 microseconds. Once all themessage processing latencies for each stage are determined, each messageprocessing latency, or the average of the message processing latenciesfor any one stage, is used to determine whether the associatedcheckpoint pair should be marked as delayed. As discussed above, thelatency detection system 508 may use a variety of differenttechniques/delay conditions to determine if message processing through astage was delayed.

The indexing system 500 then modifies the data store 700 by adding aspecial column 752, as shown in FIG. 7C. In particular, indexing system500 adds a column 752 to the data store 700, i.e., an additional datafield for each record therein.

It should be appreciated that each row in data store 700 includes a dataentry associated a checkpoint. If a checkpoint in a data entry is partof a delayed checkpoint pair (as determined by the latency detectionsystem 508), the indexing system marks a flag in the special column 752for that checkpoint's data entry. For example, if the indexing systemdetermines that message M2 experienced a delay in stage 3, then the twodata entries (720 and 724) associated with the two checkpoints definingstage 3, namely, 618 and 624, and associated with message M2 are markedwith a flag, indicated with an ‘x’, in column 752, as shown in FIG. 7C.The indexing system may similarly mark up any data entries associatedwith delayed checkpoint pairs as shown in FIG. 7C.

Accordingly, the indexing system modifies the data store 700 to includea special column 752. The timestamps of two different rows that areassociated with the same stage are used to determine whether those tworows will include a flag in column 752.

If a performance analysis tool submits a query and requests that thequery be limited to bottlenecked areas only, the data store searcher 506only searches the rows that include flags in special column 752. Forexample, a user may query the data store 700 for messages that wereprocessed between times 3 microseconds and 12 microseconds. If the queryis not limited by the user to bottlenecked areas, the system returnsrows 706, 708, 710, 712, 714, 716 and 718, or 7 rows. If, however, thequery is limited by the user to bottlenecked areas, the system onlyreturns rows 708, 714, and 718, or 3 rows. Moreover, the fields that aresearched to arrive at those search results may also be drasticallyreduced. For example, without limiting the search results to justflagged rows, the entire data store, or 12 rows, may need to besearched. But if the search is limited to flagged rows, the startingpoint of the search may be only 6 rows, namely, rows 708, 714, 718, 720,722 and 724. The disclosed indexing system accordingly enables a drasticimprovement in the speed and efficiency with which a performancemeasurement database can be queried.

FIG. 8 illustrates an example flowchart of a computer implemented method800 of indexing a data store in a data transaction processing system.Embodiments may involve all, more or fewer actions than the illustratedactions. The actions may be performed in the order or sequence shown, orin a different sequence. The actions may be performed simultaneously, orin a parallel or overlapping fashion. The method may be performed byprocessing logic that may comprise hardware (circuitry, dedicated logic,etc.), software, or a combination of both. In one example, the method isperformed by the computer system 100 of FIG. 1 , while in some otherexamples, some or all of the method may be performed by another machine.

At step 802, method 800 includes detecting, by a processor coupled withthe application, that the application has received a message of aplurality of messages for processing as each message of the plurality ofmessages is received by the application, the application including aplurality of checkpoints.

At step 804, method 800 includes, upon a received message of theplurality of messages traversing a checkpoint of the plurality ofcheckpoints, storing, by the processor, in a data store, a data entryindicative of the received message, the traversed checkpoint, and a timewhen the received message traversed the checkpoint.

At step 806, method 800 includes detecting, by the processor, that amessage processing latency between a pair of checkpoints from theplurality of checkpoints satisfies a delay condition, and based thereon,modifying the data entries associated with the pair of checkpoints byadding data indicative of a delay. For example, a latency detectionsystem may detect that one or more previously processed messagesexperienced a message processing latency when being processed from theportion of the code indicated by the first checkpoint in the pair ofcheckpoints to the portion of the code indicated by the secondcheckpoint in the pair of checkpoints. The processor, which may be partof the indexing system, may use the information provided by the latencydetection system to mark checkpoints as delayed as discussed herein.

At step 808, method 800 includes, upon receiving, by the processor, aquery that is limited to bottlenecks, searching, by the processor, onlydata entries including data indicative of a delay. For example, theprocessor may ignore all other data entries in the data store that donot include data indicative of a delay if the query is limited tobottlenecks/delays.

At step 810, method 800 includes, upon receiving, by the processor, aquery to search the entire data store, searching, by the processor, allthe data entries in the data store.

In one embodiment, the delay condition comprises one of: (i) the amountof time that elapses for a message to traverse the pair of checkpointsexceeds a threshold amount of time; (ii) the amount of time that elapsesfor a message to traverse the pair of checkpoints exceeds the amount oftime that elapses for a message to traverse a different paircheckpoints; (iii) the amount of time that elapses for a message to beprocessed by a portion of the application defined by a pair ofcheckpoints is greater than an amount of time between new messagesarriving at that portion of the application; or (iv) if a number ofmessages awaiting processing by a portion of the application defined bya pair of checkpoints exceeds a threshold number of messages.

A query that is limited to bottlenecks may be referred to as abottleneck limited query. Because bottleneck limited queries may onlysearch through data entries indicative of checkpoints marked as delayed,it should be appreciated that a bottleneck limited query is executedfaster than a query to search all of the data entries in the data store.

Queries run the data store may include or comprise determiningpercentiles associated with message processing; (ii) determining messageprocessing latency between a pair of checkpoints from the plurality ofcheckpoints over a specified timeframe.

In one embodiment, a data store may include data entries associated withthe processing of more than one application. A query may request resultsfor data entries processed by a particular application.

In one embodiment, detecting whether a message processing latencybetween a pair of checkpoints from the plurality of checkpointssatisfies a delay condition comprises determining an elapse of timebetween times when a received message traversed the two checkpoints inthe pair of checkpoints. In one embodiment, the message processinglatency between a pair of checkpoints from the plurality of checkpointsis determined based on data entries stored in the data store andindicative of times when a received message traversed the pair ofcheckpoints.

The indexing system may accordingly index the data in the data storeduring a post-processing phase and prepare the data to enable quickerand more efficient query responses. The indexing system may calculate,based on the logged data entries, message processing latencies fordifferent stages within the application, which may include calculatingthe amount of time elapsed between a message traversing checkpoint pair.The indexing system may identify the checkpoints that are associatedwith bottlenecks in the application.

It should be appreciated that the monitoring system that collects dataentries does not need to select which data entries are stored in thedata store. A developer can add as many checkpoints as necessary to theapplication, where, at each checkpoint, a data entry is recorded in thedata store. The indexing system can then determine which of the dataentries are associated with delays, and when a user of a performanceanalysis tool limits a request to bottlenecks/delays, the indexingsystem limits the search to bottlenecks, as discussed herein. Theindexing system accordingly enables a system to collect a large amountof data points, without having to search all of those data points ineach query. Thus, the indexing system allows users to maintain a highlevel of configurability and flexibility as to what data is searched,without sacrificing the ability to run fast searches (e.g., by limitingthe searches to bottlenecked/delayed checkpoints). The user could, forexample, choose to run queries against both bottlenecked checkpoints, aswell as all checkpoints, thus increasing the flexibility with whichusers can query the data store.

The illustrations of the embodiments described herein are intended toprovide a general understanding of the structure of the variousembodiments. The illustrations are not intended to serve as a completedescription of all of the elements and features of apparatus and systemsthat utilize the structures or methods described herein. Many otherembodiments may be apparent to those of skill in the art upon reviewingthe disclosure. Other embodiments may be utilized and derived from thedisclosure, such that structural and logical substitutions and changesmay be made without departing from the scope of the disclosure.Additionally, the illustrations are merely representational and may notbe drawn to scale. Certain proportions within the illustrations may beexaggerated, while other proportions may be minimized. Accordingly, thedisclosure and the figures are to be regarded as illustrative ratherthan restrictive.

While this specification contains many specifics, these should not beconstrued as limitations on the scope of the invention or of what may beclaimed, but rather as descriptions of features specific to particularembodiments of the invention. Certain features that are described inthis specification in the context of separate embodiments can also beimplemented in combination in a single embodiment. Conversely, variousfeatures that are described in the context of a single embodiment canalso be implemented in multiple embodiments separately or in anysuitable sub-combination. Moreover, although features may be describedas acting in certain combinations and even initially claimed as such,one or more features from a claimed combination can in some cases beexcised from the combination, and the claimed combination may bedirected to a sub-combination or variation of a sub-combination.

Similarly, while operations are depicted in the drawings and describedherein in a particular order, this should not be understood as requiringthat such operations be performed in the particular order shown or insequential order, or that all illustrated operations be performed, toachieve desirable results. In certain circumstances, multitasking andparallel processing may be advantageous. Moreover, the separation ofvarious system components in the described embodiments should not beunderstood as requiring such separation in all embodiments, and itshould be understood that the described program components and systemscan generally be integrated together in a single software product orpackaged into multiple software products.

One or more embodiments of the disclosure may be referred to herein,individually and/or collectively, by the term “invention” merely forconvenience and without intending to voluntarily limit the scope of thisapplication to any particular invention or inventive concept. Moreover,although specific embodiments have been illustrated and describedherein, it should be appreciated that any subsequent arrangementdesigned to achieve the same or similar purpose may be substituted forthe specific embodiments shown. This disclosure is intended to cover anyand all subsequent adaptations or variations of various embodiments.Combinations of the above embodiments, and other embodiments notspecifically described herein, will be apparent to those of skill in theart upon reviewing the description.

The Abstract of the Disclosure is provided to comply with 37 C.F.R. §1.72(b) and is submitted with the understanding that it will not be usedto interpret or limit the scope or meaning of the claims. In addition,in the foregoing Detailed Description, various features may be groupedtogether or described in a single embodiment for the purpose ofstreamlining the disclosure. This disclosure is not to be interpreted asreflecting an intention that the claimed embodiments require morefeatures than are expressly recited in each claim. Rather, as thefollowing claims reflect, inventive subject matter may be directed toless than all of the features of any of the disclosed embodiments. Thus,the following claims are incorporated into the Detailed Description,with each claim standing on its own as defining separately claimedsubject matter.

It is therefore intended that the foregoing detailed description beregarded as illustrative rather than limiting, and that it be understoodthat it is the following claims, including all equivalents, that areintended to define the spirit and scope of this invention.

1. A computer implemented method comprising: evaluating, by a processor,a data store coupled therewith, the data store storing data entriestracing a progress of a traversal of each of a plurality of messagesthrough a plurality of checkpoints of an application operative toprocess the plurality of messages upon a receipt of each thereof;determining, by the processor, a plurality of stages, each of theplurality of stages defined by a pair of checkpoints of the plurality ofcheckpoints; detecting, by the processor, that a message processinglatency of one or more of the plurality of messages through a stage ofthe plurality of stages satisfies a delay condition, and based thereon,modifying the data entries associated with the pair of checkpointsdefining the stage to indicate a delay; and upon receiving, by theprocessor, a query that is limited to bottlenecks, searching, only dataentries including data indicative of the delay.
 2. The computerimplemented method of claim 1, wherein the delay condition comprises oneof: (i) an amount of time that elapses for a message to traverse thestage exceeds a threshold amount of time; (ii) the amount of time thatelapses for a message to traverse the stage exceeds the amount of timethat elapses for a message to traverse a different stage; (iii) theamount of time that elapses for a message to be processed by a portionof the application defined by the stage is greater than an amount oftime between new messages arriving at that portion of the application;or (iv) if a number of messages awaiting processing by a portion of theapplication defined by the stage exceeds a threshold number of messages.3. The computer implemented method of claim 1, further comprising, uponreceipt of a query to search the entire data store, searching, by theprocessor, all the data entries in the data store.
 4. The computerimplemented method of claim 3, wherein the bottleneck limited query isexecuted faster than the query to search all the data entries.
 5. Thecomputer implemented method of claim 1, wherein the detecting furthercomprises determining an elapse of time when a received messagetraversed the stage.
 6. The computer implemented method of claim 1,further comprising determining the message processing latency throughthe stage based on the data entries stored in the data store andindicative of times when the one or more messages traversed the stage.7. The computer implemented method of claim 1, wherein the messageprocessing latency of the stage is determined based on the data entriesstored in the data store during the processing thereof of the pair ofcheckpoints.
 8. The computer implemented method of claim 1, wherein thequery comprises one or more of: (i) determining percentiles associatedwith message processing; and (ii) determining the message processinglatency through the stage over a specified period of time.
 9. Thecomputer implemented method of claim 1, wherein each message is anelectronic data transaction request message, and wherein the applicationprocesses an electronic data transaction request message by determiningwhether an attempt to match an electronic data transaction requestmessage with at least one previously received but unsatisfied electronicdata transaction request message for a transaction which is counterthereto results in at least partial satisfaction of one or both of theelectronic data transaction request message and the at least onepreviously received but unsatisfied electronic data transaction requestmessage.
 10. The computer implemented method of claim 1, wherein theevaluating occurs only after all of the plurality of messages have beenprocessed by the application.
 11. The computer implemented method ofclaim 1, wherein the query is received only after all of the pluralityof messages have been processed by the application.
 12. The computerimplemented method of claim 1, wherein the pair of checkpoints are notconsecutive.
 13. The computer implemented method of claim 1, furthercomprising modifying, by the processor, all of the data entries betweenthe pair of checkpoints defining the stage to indicate a delay.
 14. Ameasurement indexing system comprising: a processor; a data storecoupled with the processor, the data store configured to store dataentries tracing a progress of a traversal of each of a plurality ofmessages through a plurality of checkpoints of an application operativeto process the plurality of messages upon a receipt of each thereof; anon-transitory computer-readable medium coupled with the processor, thenon-transitory computer readable medium storing instructions which causethe processor to: evaluate the data entries stored in the data store;determine, a plurality of stages, each of the plurality of stagesdefined by a pair of checkpoints of the plurality of checkpoints; detectthat a message processing latency of one or more of the plurality ofmessages through a stage of the plurality of stages satisfies a delaycondition, and based thereon, modify the data entries associated withthe pair of checkpoints defining the stage to indicate a delay; and uponreceipt of a query that is limited to bottlenecks, search only dataentries including data indicative of the delay.
 15. The measurementindexing system of claim 14, wherein the delay condition comprises oneof: (i) an amount of time that elapses for a message to traverse thestage exceeds a threshold amount of time; (ii) the amount of time thatelapses for a message to traverse the stage exceeds the amount of timethat elapses for a message to traverse a different stage; (iii) theamount of time that elapses for a message to be processed by a portionof the application defined by the stage is greater than an amount oftime between new messages arriving at that portion of the application;or (iv) if a number of messages awaiting processing by a portion of theapplication defined by the stage exceeds a threshold number of messages.16. The measurement indexing system of claim 14, wherein theinstructions further cause the processor to: upon receipt of a query tosearch the entire data store, search all the data entries in the datastore.
 17. The measurement indexing system of claim 16, wherein thebottleneck limited query is faster than the query to search all of thedata entries.
 18. The measurement indexing system of claim 14, whereinthe detection of whether the message processing latency through thestage satisfies the delay condition comprises determination of an elapseof time between times when the one or more messages traversed the stage.19. The measurement indexing system of claim 14, wherein theinstructions are further configured to cause the processor to determinethe message processing latency through the stage based on the dataentries stored in the data store and indicative of times when the one ormore messages traversed the stage.
 20. The measurement indexing systemof claim 14, wherein the query comprises one or more of: (i) adetermination of percentiles associated with message processing; and(ii) a determination of the message processing latency through the stageover a specified timeframe.
 21. The indexing system of claim 14, whereineach message is an electronic data transaction request message, andwherein the application processes an electronic data transaction requestmessage by determining whether an attempt to match an electronic datatransaction request message with at least one previously received butunsatisfied electronic data transaction request message for atransaction which is counter thereto results in at least partialsatisfaction of one or both of the electronic data transaction requestmessage and the at least one previously received but unsatisfiedelectronic data transaction request message.
 22. The measurementindexing system of claim 14, wherein the evaluation occurs only afterall of the plurality of messages have been processed by the application.23. The measurement indexing system of claim 14, wherein the query isreceived only after all of the plurality of messages have been processedby the application.
 24. The measurement indexing system of claim 14,wherein the pair of checkpoints are not consecutive.
 25. The measurementindexing system of claim 14, wherein the instructions are furtherconfigured to cause the processor to modify all of the data entriesbetween the pair of checkpoints defining the stage to indicate a delay.26. A measurement indexing system comprising: means for evaluating adata store, which stores data entries tracing a progress of a traversalof each of a plurality of messages through a plurality of checkpoints ofan application operative to process the plurality of messages upon areceipt of each thereof; and means for determining a plurality ofstages, each of the plurality of stages defined by a pair of checkpointsof the plurality of checkpoints; means for detecting that a messageprocessing latency of one or more of the plurality of messages through astage of the plurality of stages satisfies a delay condition, and basedthereon, modifying the data entries associated with the pair ofcheckpoints defining the stage to indicate a delay; and means forsearching, upon receiving a query that is limited to bottlenecks, onlydata in the data store which indicates the delay.